TimberWinR

A Native Windows to Redis Logstash Agent which runs as a service.

Why have TimberWinR?

TimberWinR is a native .NET implementation utilizing Microsoft's LogParser. This means no JVM/JRuby is required, and LogParser does all the heavy lifting. TimberWinR collects the data from LogParser and ships it to Logstash via Redis.

Basics

TimberWinR uses a configuration file to control how the logs are collected, filtered and shipped off.
These are broken down into:

1. Inputs (Collect data from different sources)
2. Filters (Are applied to all Inputs)
3. Outputs (Currently ships only to Redis)

Input Formats

The current supported Input format sources are:

1. Logs (Files, a.k.a Tailing a file)
2. Tcp (listens on a port for JSON messages)
3. IISW3C(Internet Information Services W3C Format)
4. WindowsEvents (Windows Event Viewer)
5. Stdin (Standard Input for Debugging)

Filters

The current list of supported filters are:

1. Grok
2. Mutate
3. Date

JSON

Since TimberWinR only ships to Redis, the format generated by TimberWinR is JSON. All fields referenced by TimberWinR can be represented as a JSON Property or Array.

Supported Output Formats

1. Redis

Sample Configuration

TimberWinR reads a JSON configuration file, an example file is shown here:

{
"TimberWinR": {
    "Inputs": {
        "WindowsEvents": [
            {
                "source": "System,Application",
                "binaryFormat": "PRINT",
                "resolveSIDS": true
            }
        ]
    },
    "Filters": [          
        {
            "grok": {
                "condition": "[type] == \"Win32-Eventlog\"",
                "match": [
                    "Message",
                    ""
                ],                   
                "remove_field": [
                    "ComputerName"                   
                ]
            }
        }
    ],
    "Outputs": {
        "Redis": [
            { 
                "_comment": "Shuffle these hosts",
                "host": [
                   "server1.host.com", 
                   "server2.host.com"
                ]
            }
        ]
    }
}

This configuration:

1. Inputs: Events from the Windows Event Logs (System, Application)
2. Filters: Removes the ComputerName field
3. Sends the event to Redis services (server1.host.com, server2.host.com) in a shuffling manner (balanced).

Installation

You must first install LogParser, then install TimberWinR. Install LogParser from here:

Install LogParser from Microsoft.

After installing, follow the remaining directions here.

Running Interactively

You can run TimberWinR interactively when you are developing your JSON config file, to do so use the following options:

TimberWinR.ServiceHost.exe -configFile:myconfig.json -logLevel:Debug

Installation as a Windows Service

TimberWinR uses TopShelf to install as a service, so all the documentation for installing and configuring the service is show here TopShelf Doc

Specifically the command line options are listed here in Topshelf Command-Line Reference guide.

Install and set to Automatically Start the service:

; Install Service (will autostart on reboot)
TimberWinR.ServiceHost.exe install --autostart
; Start the Service
TimberWinR.ServiceHost.exe start

To Start/Stop the Service from the Command Line

TimberWinR.ServiceHost.exe start
TimberWinR.ServiceHost.exe stop

Alternatively you can use the Services Control Panel.

Usage

TimberWinR.ServiceHost.exe [options]

Options:
-logDir:        Specifies the directory where TimberWinR will write its log file TimberWinR.txt
                Default is -logDir:"C:\logs"
-logLevel:      Specifies the logging level for TimberWinR
                Legal Values: Trace|Debug|Info|Warn|Error|Fatal|Off
                Default is -logDir:Info
-configFile:    Specifies the path to the JSON config files
                Default is -configFile:default.json