TimberWinR

A Native Windows to Redis Logstash Agent which runs as a service.

Why have TimberWinR?

TimberWinR is a native .NET implementation utilizing Microsoft's LogParser. This means no JVM/JRuby is required, and LogParser does all the heavy lifting. TimberWinR collects the data from LogParser and ships it to Logstash via Redis.

Basics

TimberWinR uses a configuration file to control how the logs are collected, filtered and shipped off.
These are broken down into:

1. Inputs (Collect data from different sources)
2. Filters (Are applied to all Inputs)
3. Outputs (Currently ships only to Redis)

Input Formats

The current supported Input format sources are:

1. Logs (Files, a.k.a Tailing a file)
2. Tcp (listens on a port for JSON messages)
3. IISW3C(Internet Information Services W3C Format)
4. WindowsEvents (Windows Event Viewer)

Filters

The current list of supported filters are:

1. Grok
2. Mutate
3. Date

JSON

Since TimberWinR only ships to Redis, the format generated by TimberWinR is JSON. All fields referenced by TimberWinR can be represented as a JSON Property or Array.

Supported Output Formats

1. Redis

Sample Configuration

TimberWinR reads a JSON configuration file, an example file is shown here:

{
"TimberWinR": {
    "Inputs": {
        "WindowsEvents": [
            {
                "source": "System,Application",
                "binaryFormat": "PRINT",
                "resolveSIDS": true
            }
        ]
    },
    "Filters": [          
        {
            "grok": {
                "condition": "[type] == \"Win32-Eventlog\"",
                "match": [
                    "Message",
                    ""
                ],                   
                "remove_field": [
                    "ComputerName"                   
                ]
            }
        }
    ],
    "Outputs": {
        "Redis": [
            { 
                "_comment": "Shuffle these hosts",
                "host": [
                   "server1.host.com", 
                   "server2.host.com"
                ]
            }
        ]
    }
}

This configuration:

1. Inputs: Events from the Windows Event Logs (System, Application)
2. Filters: Removes the ComputerName field
3. Sends the event to Redis services (server1.host.com, server2.host.com) in a shuffling manner (balanced).

Installation

You must first install LogParser, then install TimberWinR. Install LogParser from here:

Install LogParser from Microsoft.

After installing, follow the remaining directions here.

Running Interactively

TimberWinR.ServiceHost.exe -configFile:myconfig.json -logLevel:Debug

Installation as a Windows Service

TimberWinR uses TopShelf to install as a service, so all the documentation for installing and configuring the service is show here TopShelf Doc

Specifically the command line options are listed here in Topshelf Command-Line Reference guide.

Install and set to Automatically Start the service:

; Install Service (will autostart on reboot)
TimberWinR.ServiceHost.exe install --autostart
; Start the Service
TimberWinR.ServiceHost.exe start

To Start/Stop the Service from the Command Line

TimberWinR.ServiceHost.exe start
TimberWinR.ServiceHost.exe stop

Alternatively you can use the Services Control Panel.

Usage

TimberWinR.ServiceHost.exe [options]

Options:
-logDir:        Specifies the directory where TimberWinR will write its log file TimberWinR.txt
                Default is -logDir:"C:\logs"
-logLevel:      Specifies the logging level for TimberWinR
                Legal Values: Trace|Debug|Info|Warn|Error|Fatal|Off
                Default is -logDir:Info
-configFile:    Specifies the path to the JSON config files
                Default is -configFile:default.json