change DEPRECATION notice

Update README.md
Add deprecation notice.
2016-01-20 08:06:25 -05:00 · 2015-11-05 14:53:57 -05:00 · 2015-09-11 06:46:29 -04:00 · 2015-08-31 17:15:36 -07:00 · 2015-07-13 11:26:50 -04:00 · 2015-06-19 11:47:45 -04:00
59 changed files with 3757 additions and 176 deletions
--- a/.nuget/packages.config
+++ b/.nuget/packages.config
@@ -2,6 +2,6 @@
 <packages>
  <package id="Newtonsoft.Json" version="6.0.4" targetFramework="net40" />
  <package id="NUnit.Runners" version="2.6.4" />
-  <package id="RapidRegex.Core" version="1.0.0.2" targetFramework="net40" />
+  <package id="RapidRegex.Core" version="1.0.0.4" targetFramework="net40" />
  <package id="System.Linq.Dynamic" version="1.0.4" targetFramework="net40" />
 </packages>
--- a/README.md
+++ b/README.md
@@ -2,6 +2,14 @@ TimberWinR
 ==========
 A Native Windows to Redis/Elasticsearch Logstash Agent which runs as a service.

+## Development and Roadmap
+
+TimberWinR's development has been and in the short term will continue to be on a hiatus. The project's ownership has been recently transferred, and thus the project will be slow to pick up steam again. Currently a roadmap is being established, and a vision for the project is currently being formulated.
+
+#### Getting Involved
+
+If you wish to be involved with TimberWinR in an on-going basis file an issue.
+
 ## Why have TimberWinR?
 TimberWinR is a native .NET implementation utilizing Microsoft's [LogParser](http://technet.microsoft.com/en-us/scriptcenter/dd919274.aspx).  This means
 no JVM/JRuby is required, and LogParser does all the heavy lifting.  TimberWinR collects
@@ -9,14 +17,14 @@ the data from LogParser and ships it to Logstash via Redis (or can ship direcly

 ## Release Notes

-[View Version History](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/ReleaseNotes.md)
+[View Version History](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/ReleaseNotes.md)

 ## Basics
 TimberWinR uses a configuration file to control how the logs are collected, filtered and shipped off.  
 These are broken down into:
 1. Inputs  (Collect data from different sources)
 2. Filters (Are applied to all Inputs)
- 3. Outputs (Redis, Elasticsearch or Stdout)
+ 3. Outputs (e.g. Redis, Elasticsearch, Stdout, StatsD)

 ### Support ###
 Please use the TimberWinR Google Group for discussion and support:
@@ -30,27 +38,27 @@ Latest Build:

 ## Inputs
 The current supported Input format sources are:
- 1. [Logs](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/Logs.md) (Files, a.k.a Tailing a file)
- 2. [Tcp](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/TcpInput.md) (listens on TCP port for JSON messages)
- 3. [IISW3C](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/IISW3CInput.md)(Internet Information Services W3C Format)
- 4. [WindowsEvents](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/WindowsEvents.md) (Windows Event Viewer)
- 5. [Stdin](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/StdinInput.md) (Standard Input for Debugging)
- 6. [W3C](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/W3CInput.md)(Internet Information Services W3C Advanced/Custom Format)
- 7. [Udp](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/UdpInput.md) (listens for UDP on port for JSON messages)
- 8. [TailFiles](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/TailFiles.md) (Tails log files efficiently)
- 8. [Generator](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/Generator.md) (Generate logs for testing *New*)
+ 1. [Logs](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/Logs.md) (Files, a.k.a Tailing a file)
+ 2. [Tcp](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/TcpInput.md) (listens on TCP port for JSON messages)
+ 3. [IISW3C](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/IISW3CInput.md)(Internet Information Services W3C Format)
+ 4. [WindowsEvents](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/WindowsEvents.md) (Windows Event Viewer)
+ 5. [Stdin](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/StdinInput.md) (Standard Input for Debugging)
+ 6. [W3C](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/W3CInput.md)(Internet Information Services W3C Advanced/Custom Format)
+ 7. [Udp](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/UdpInput.md) (listens for UDP on port for JSON messages)
+ 8. [TailFiles](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/TailFiles.md) (Tails log files efficiently)
+ 8. [Generator](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/Generator.md) (Generate logs for testing *New*)

 ## Codecs
 The current list of supported codecs are:
- 1. [Multiline](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/Codec.md)
+ 1. [Multiline](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/Codec.md)

 ## Filters
 The current list of supported filters are:
- 1. [Grok](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/GrokFilter.md)
- 2. [Mutate](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/MutateFilter.md)
- 3. [Date](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/DateFilter.md)
- 4. [Json](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/JsonFilter.md)
- 5. [GeoIP](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/GeoIPFilter.md)
+ 1. [Grok](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/GrokFilter.md)
+ 2. [Mutate](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/MutateFilter.md)
+ 3. [Date](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/DateFilter.md)
+ 4. [Json](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/JsonFilter.md)
+ 5. [GeoIP](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/GeoIPFilter.md)

 Note that there are now two syntaxes for filters, array and single, if you want more than one filter of the same
 type then you must use the array tag instead of the singular tag. i.e:
@@ -104,9 +112,11 @@ Since TimberWinR only ships to Redis and Elasticsearch, the format generated by
 represented as a JSON Property or Array.

 ## Outputs
-1. [Redis](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/RedisOutput.md)
-2. [Elasticsearch](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/ElasticsearchOutput.md)
-3. [Stdout](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/StdoutOutput.md)
+1. [Redis](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/RedisOutput.md)
+2. [Elasticsearch](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/ElasticsearchOutput.md)
+3. [Stdout](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/StdoutOutput.md)
+4. [File](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/FileOutput.md)
+5. [StatsD](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/StatsD.md)

 ## Sample Configuration
 TimberWinR reads a JSON configuration file, an example file is shown here:
--- a/TimberWinR.ServiceHost/Program.cs
+++ b/TimberWinR.ServiceHost/Program.cs
@@ -25,11 +25,11 @@ namespace TimberWinR.ServiceHost
    {
        const string KeyPath = @"HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\TimberWinR";
        const string KeyName = "ImagePath";
-        
+
        private static void Main(string[] args)
        {
            Arguments arguments = new Arguments();
-           
+
            HostFactory.Run(hostConfigurator =>
            {
                string cmdLine = Environment.CommandLine;
@@ -45,7 +45,7 @@ namespace TimberWinR.ServiceHost
                hostConfigurator.AddCommandLineDefinition("configFile", c => arguments.ConfigFile = c);
                hostConfigurator.AddCommandLineDefinition("logLevel", c => arguments.LogLevel = c);
                hostConfigurator.AddCommandLineDefinition("logDir", c => arguments.LogfileDir = c);
-                hostConfigurator.AddCommandLineDefinition("diagnosticPort", c => arguments.DiagnosticPort = int.Parse(c));    
+                hostConfigurator.AddCommandLineDefinition("diagnosticPort", c => arguments.DiagnosticPort = int.Parse(c));

                hostConfigurator.ApplyCommandLine();
                hostConfigurator.RunAsLocalSystem();
@@ -56,7 +56,7 @@ namespace TimberWinR.ServiceHost
                hostConfigurator.SetServiceName("TimberWinR");

                hostConfigurator.AfterInstall(() =>
-                { 
+                {
                    var currentValue = Registry.GetValue(KeyPath, KeyName, "").ToString();
                    if (!string.IsNullOrEmpty(currentValue))
                    {
@@ -72,18 +72,18 @@ namespace TimberWinR.ServiceHost
        }

        private static void AddServiceParameter(string paramName, string value)
-        {         
+        {
            string currentValue = Registry.GetValue(KeyPath, KeyName, "").ToString();

-            if (!string.IsNullOrEmpty(paramName) && !currentValue.Contains(string.Format("{0} ", paramName)))           
+            if (!string.IsNullOrEmpty(paramName) && !currentValue.Contains(string.Format("{0} ", paramName)))
            {
-                currentValue += string.Format(" {0} \"{1}\"", paramName, value.Replace("\\\\", "\\"));               
-                Registry.SetValue(KeyPath, KeyName, currentValue);               
+                currentValue += string.Format(" {0} \"{1}\"", paramName, value.Replace("\\\\", "\\"));
+                Registry.SetValue(KeyPath, KeyName, currentValue);
            }
        }

        private static void AddServiceParameter(string paramName, int value)
-        {    
+        {
            string currentValue = Registry.GetValue(KeyPath, KeyName, "").ToString();

            if (!string.IsNullOrEmpty(paramName) && !currentValue.Contains(string.Format("{0}:", paramName)))
@@ -131,13 +131,15 @@ namespace TimberWinR.ServiceHost
        private readonly Arguments _args;
        private TimberWinR.Diagnostics.Diagnostics _diags;
        private TimberWinR.Manager _manager;
+        public bool StartingUp { get; set; }
+        public bool Started { get; set; }

        public TimberWinRService(Arguments args)
        {
            _args = args;
            _cancellationTokenSource = new CancellationTokenSource();
            _cancellationToken = _cancellationTokenSource.Token;
-            _serviceTask = new Task(RunService, _cancellationToken);           
+            _serviceTask = new Task(RunService, _cancellationToken);
        }

        public void Start()
@@ -147,22 +149,41 @@ namespace TimberWinR.ServiceHost

        public void Stop()
        {
+            WaitForStartupToComplete();
+
            _cancellationTokenSource.Cancel();
            if (_diags != null)
-             _diags.Shutdown();
+                _diags.Shutdown();

            if (_manager != null)
                _manager.Shutdown();
        }

+        // If you bounce the service too quickly, the shutdown can occur
+        // before the service has started, which results in a hang, this blocks until 
+        // all thread have properly started (waiting up to 10 seconds max) 
+        private void WaitForStartupToComplete()
+        {
+            int tries = 100; // 10 seconds max
+            if (StartingUp)
+            {
+                while (!Started && tries-- >= 0)
+                {                  
+                    Thread.Sleep(100);
+                }
+            }
+        }
+
        /// <summary>
        /// The Main body of the Service Worker Thread
        /// </summary>
        private void RunService()
        {
+            StartingUp = true;
            _manager = new TimberWinR.Manager(_args.ConfigFile, _args.LogLevel, _args.LogfileDir, _args.LiveMonitor, _cancellationToken);
            if (_args.DiagnosticPort > 0)
                _diags = new Diagnostics.Diagnostics(_manager, _cancellationToken, _args.DiagnosticPort);
+            Started = true;
        }
    }
 }
--- a/TimberWinR.ServiceHost/Properties/AssemblyInfo.cs
+++ b/TimberWinR.ServiceHost/Properties/AssemblyInfo.cs
@@ -32,5 +32,5 @@ using System.Runtime.InteropServices;
 // You can specify all the values or you can default the Build and Revision Numbers 
 // by using the '*' as shown below:
 // [assembly: AssemblyVersion("1.0.*")]
-[assembly: AssemblyVersion("1.3.24.0")]
-[assembly: AssemblyFileVersion("1.3.24.0")]
+[assembly: AssemblyVersion("1.3.26.0")]
+[assembly: AssemblyFileVersion("1.3.26.0")]
--- a/TimberWinR.ServiceHost/TimberWinR.ServiceHost.csproj
+++ b/TimberWinR.ServiceHost/TimberWinR.ServiceHost.csproj
@@ -38,9 +38,16 @@
    <ApplicationIcon>timberwinr.ico</ApplicationIcon>
  </PropertyGroup>
  <ItemGroup>
-    <Reference Include="RapidRegex.Core, Version=1.0.0.0, Culture=neutral, processorArchitecture=MSIL">
+    <Reference Include="RapidRegex.Core, Version=1.0.0.4, Culture=neutral, processorArchitecture=MSIL">
      <SpecificVersion>False</SpecificVersion>
-      <HintPath>..\packages\RapidRegex.Core.1.0.0.2\lib\net40\RapidRegex.Core.dll</HintPath>
+      <HintPath>..\packages\RapidRegex.Core.1.0.0.4\lib\net40\RapidRegex.Core.dll</HintPath>
+    </Reference>
+    <Reference Include="StatsdClient, Version=1.0.0.19, Culture=neutral, processorArchitecture=MSIL">
+      <SpecificVersion>False</SpecificVersion>
+      <HintPath>..\packages\StatsdClient.1.0.0.19\lib\net35\StatsdClient.dll</HintPath>
+    </Reference>
+    <Reference Include="StatsdClient.Configuration">
+      <HintPath>..\packages\StatsdClient.1.0.0.19\lib\net35\StatsdClient.Configuration.dll</HintPath>
    </Reference>
    <Reference Include="System" />
    <Reference Include="System.Configuration.Install" />
--- a/TimberWinR.ServiceHost/packages.config
+++ b/TimberWinR.ServiceHost/packages.config
@@ -1,5 +1,6 @@
 <?xml version="1.0" encoding="utf-8"?>
 <packages>
-  <package id="RapidRegex.Core" version="1.0.0.2" targetFramework="net40" />
+  <package id="RapidRegex.Core" version="1.0.0.4" targetFramework="net40" />
+  <package id="StatsdClient" version="1.0.0.19" targetFramework="net40" />
  <package id="Topshelf" version="3.1.4" targetFramework="net40" />
 </packages>
--- a/TimberWinR.TestGenerator/CommandLineOptions.cs
+++ b/TimberWinR.TestGenerator/CommandLineOptions.cs
@@ -27,6 +27,9 @@ namespace TimberWinR.TestGenerator
        [Option("resultsFile", HelpText = "Expected results Results json file")]
        public string ExpectedResultsFile { get; set; }

+        [Option("totalMessages", DefaultValue = 0, HelpText = "The total number of messages to send to the output(s)")]
+        public int TotalMessages { get; set; }
+
        [Option('n', "numMessages", DefaultValue = 1000, HelpText = "The number of messages to send to the output(s)")]
        public int NumMessages { get; set; }

--- a/TimberWinR.TestGenerator/Program.cs
+++ b/TimberWinR.TestGenerator/Program.cs
@@ -350,7 +350,7 @@ namespace TimberWinR.TestGenerator
                    var mbc = outputToken["queuedMessageCount"].Value<int>();
                    var smc = outputToken["sentMessageCount"].Value<int>();

-                    // LogManager.GetCurrentClassLogger().Info("Queued: {0}, Sent: {1}", mbc, smc);
+                    //LogManager.GetCurrentClassLogger().Info("Output: {2} Queued: {0}, Sent: {1}", mbc, smc, outputToken.ToString());

                    completed = mbc == 0 && smc >= _totalMessagesToSend;
                }
@@ -504,6 +504,8 @@ namespace TimberWinR.TestGenerator

        static Task[] RunGenerators(CommandLineOptions options)
        {
+            _totalMessagesToSend = options.TotalMessages;
+
            _monitorTask = Task.Factory.StartNew(() =>
                {
                    using (var syncHandle = new ManualResetEventSlim())
--- a/TimberWinR.TestGenerator/TimberWinR.TestGenerator.csproj
+++ b/TimberWinR.TestGenerator/TimberWinR.TestGenerator.csproj
@@ -117,6 +117,27 @@
    <Content Include="test4.json">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </Content>
+    <Content Include="test5-twconfig.json">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+    </Content>
+    <Content Include="test5.json">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+    </Content>
+    <Content Include="results5.json">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+    </Content>
+    <Content Include="test7-tw.json">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+    </Content>
+    <Content Include="test7.json">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+    </Content>
+    <Content Include="results7.json">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+    </Content>
+    <None Include="sample-apache.log">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+    </None>
  </ItemGroup>
  <ItemGroup>
    <ProjectReference Include="..\TimberWinR\TimberWinR.csproj">
--- a/TimberWinR.TestGenerator/UdpTestGenerator.cs
+++ b/TimberWinR.TestGenerator/UdpTestGenerator.cs
@@ -1,4 +1,6 @@
-using System.Threading;
+using System.Security.Cryptography;
+using System.Threading;
+using Newtonsoft.Json;
 using Newtonsoft.Json.Linq;
 using NLog;
 using NLog.Config;
@@ -24,7 +26,7 @@ namespace TimberWinR.TestGenerator
            NumMessages = 100;
            Port = 6379;
            Host = "localhost";
-            SleepTimeMilliseconds = 10;
+            SleepTimeMilliseconds = 1;
        }
    }

@@ -48,16 +50,37 @@ namespace TimberWinR.TestGenerator
            {
                JObject o = new JObject
                {
-                    {"Application", "udp-generator"},               
-                    {"Host", hostName},
+                    {"Application", "udp-generator"},  
+                    {"Executable", "VP.Common.SvcFrm.Services.Host, Version=29.7.0.0, Culture=neutral, PublicKeyToken=null"},
+                    {"RenderedMessage", "Responding to RequestSchedule message from 10.1.230.36 with Ack because: PRJ byte array is null."},
+                    {"Team", "Manufacturing Software"},   
+                    {"RecordNumber", i},
+                    {"Host", hostName},                   
                    {"UtcTimestamp", DateTime.UtcNow.ToString("o")},
-                    {"Type", "udp"},                
+                    {"Type", "VP.Fulfillment.Direct.Initialization.LogWrapper"},                
                    {"Message", "Testgenerator udp message " + DateTime.UtcNow.ToString("o")},
                    {"Index", "logstash"}
                };
-                byte[] sendbuf = Encoding.UTF8.GetBytes(o.ToString());
+
+                string hashedString = "";
+                foreach(var key in o)
+                {
+                    hashedString += key.ToString();
+                }
+
+                var source = ASCIIEncoding.ASCII.GetBytes(hashedString);
+                var md5 = new MD5CryptoServiceProvider().ComputeHash(source);
+                var hash = string.Concat(md5.Select(x => x.ToString("X2")));
+
+                o["md5"] = hash;
+
+                byte[] sendbuf = Encoding.UTF8.GetBytes(o.ToString(Formatting.None));
                IPEndPoint ep = new IPEndPoint(broadcast, parms.Port);
                s.SendTo(sendbuf, ep);
+
+                if (i % 1000 == 0)
+                    LogManager.GetCurrentClassLogger().Info("Sent {0} of {1} messages", i, parms.NumMessages);
+
                Thread.Sleep(parms.SleepTimeMilliseconds);
            }

--- a/TimberWinR.TestGenerator/default.json
+++ b/TimberWinR.TestGenerator/default.json
@@ -36,7 +36,7 @@
                    "batch_count": 500,
                    "threads":  2,
                    "host": [
-                        "tstlexiceapp006.vistaprint.svc"
+                        "tstlexiceapp006.mycompany.svc"
                    ]
                }
            ]
--- a/TimberWinR.TestGenerator/results5.json
+++ b/TimberWinR.TestGenerator/results5.json
@@ -0,0 +1,13 @@
+{
+    "Results": {
+        "Inputs": [           
+            {
+                "udp": {
+                    "test1: message sent count": "[messages] == 80000",
+                    "test2: average cpu": "[avgCpuUsage] <= 30",
+                    "test3: maximum memory": "[maxMemUsage] <= 30"              
+                }
+            }
+        ]
+    }
+}
--- a/TimberWinR.TestGenerator/results7.json
+++ b/TimberWinR.TestGenerator/results7.json
@@ -0,0 +1,13 @@
+{
+    "Results": {
+        "Inputs": [           
+            {
+                "udp": {
+                    "test1: message sent count": "[messages] == 80000",
+                    "test2: average cpu": "[avgCpuUsage] <= 30",
+                    "test3: maximum memory": "[maxMemUsage] <= 30"              
+                }
+            }
+        ]
+    }
+}
--- a/TimberWinR.TestGenerator/sample-apache.log
+++ b/TimberWinR.TestGenerator/sample-apache.log
--- a/TimberWinR.TestGenerator/test1-twconfig.json
+++ b/TimberWinR.TestGenerator/test1-twconfig.json
@@ -39,9 +39,9 @@
                    "_comment": "Change the host to your Redis instance",
                    "port": 6379,
                    "batch_count": 500,
-                    "threads": 2,
+                    "threads": 1,
                    "host": [
-                        "tstlexiceapp006.vistaprint.svc"
+                        "tstlexiceapp006.mycompany.svc"
                    ]
                }
            ]
--- a/TimberWinR.TestGenerator/test2-tw.json
+++ b/TimberWinR.TestGenerator/test2-tw.json
@@ -36,7 +36,7 @@
                    "batch_count": 500,
                    "threads":  2,
                    "host": [
-                        "tstlexiceapp006.vistaprint.svc"
+                        "tstlexiceapp006.mycompany.svc"
                    ]
                }
            ]
--- a/TimberWinR.TestGenerator/test2.json
+++ b/TimberWinR.TestGenerator/test2.json
@@ -1,11 +1,12 @@
 {
    "test": "Test 2",
    "arguments": {
+        "--start":  "",
        "--testFile": "test2.json",
        "--testDir": "test2",
        "--timberWinRConfig": "test2-tw.json",
        "--numMessages": 1234,
-        "--logLevel": "trace",
+        "--logLevel": "debug",
        "--udp": "5140",
        "--jroll": ["r1.jlog", "r2.jlog"],
        "--json": ["1.jlog", "2.jlog", "3.jlog", "4.jlog"],
--- a/TimberWinR.TestGenerator/test3-tw.json
+++ b/TimberWinR.TestGenerator/test3-tw.json
@@ -41,7 +41,7 @@
                    "batch_count": 500,
                    "threads": 2,
                    "host": [
-                        "tstlexiceapp006.vistaprint.svc"
+                        "tstlexiceapp006.mycompany.svc"
                    ]
                }
            ]
--- a/TimberWinR.TestGenerator/test4-tw.json
+++ b/TimberWinR.TestGenerator/test4-tw.json
@@ -41,7 +41,7 @@
                    "batch_count": 500,
                    "threads": 2,
                    "host": [
-                        "tstlexiceapp006.vistaprint.svc"
+                        "tstlexiceapp006.mycompany.svc"
                    ]
                }
            ]
--- a/TimberWinR.TestGenerator/test5-twconfig.json
+++ b/TimberWinR.TestGenerator/test5-twconfig.json
@@ -0,0 +1,36 @@
+{
+    "TimberWinR": {
+        "Inputs": {
+            "Udp": [
+                {
+                    "_comment": "Output from NLog",
+                    "port": 5140,
+                    "add_field": [
+                        "Environment",
+                        "PLANT_TST_TIMBERWINR"
+                    ],
+                    "rename": [
+                        "Type",
+                        "type"
+                    ]
+                }
+            ],
+            "TailFiles": [
+                {
+                    "interval": 5,
+                    "logSource": "log files",
+                    "location": "*.jlog",
+                    "recurse": -1
+                }
+            ]
+        },
+        "Outputs": {
+            "File": [
+                {
+                    "format":  "indented",
+                    "file_name": "test5_output.txt"
+                }
+            ]
+        }
+    }
+}
--- a/TimberWinR.TestGenerator/test5.json
+++ b/TimberWinR.TestGenerator/test5.json
@@ -0,0 +1,15 @@
+{
+    "test": "Test 5",
+    "arguments": {
+        "--start":  "",
+        "--testFile": "test5.json",
+        "--testDir": "test5",
+        "--timberWinRConfig": "test5-twconfig.json",
+        "--numMessages": 80000,
+        "--logLevel": "debug",
+        "--udp-host":  "localhost",
+        "--udp": "5140",  
+        "--udp-rate":  1,            
+        "--resultsFile": "results5.json"
+    }
+}
--- a/TimberWinR.TestGenerator/test7-tw.json
+++ b/TimberWinR.TestGenerator/test7-tw.json
@@ -0,0 +1,37 @@
+{
+    "TimberWinR": {
+        "Inputs": {
+            "TailFiles": [
+                {
+                    "interval": 5,
+                    "logSource": "apache log files",
+                    "location": "..\\sample-apache.log",
+                    "recurse": -1
+                }
+            ]
+        },
+        "Filters": [
+            {
+                "grok": {
+                    "type": "Win32-TailLog",
+                    "match": [
+                        "Text",
+                        "%{COMBINEDAPACHELOG}"
+                    ]
+                }
+            }
+        ],
+        "Outputs": {            
+            "StatsD": [
+                {
+                    "type": "Win32-TailLog",
+                    "namespace": "timberwinrtest",
+                    "port": 8125,
+                    "host": "devlexicesnu003.mycompany.svc",
+                    "increment":  ["apache.response.%{response}"],
+                    "count":  ["apache.bytes", "%{bytes}"]
+                }
+            ]
+        }
+    }
+}
--- a/TimberWinR.TestGenerator/test7.json
+++ b/TimberWinR.TestGenerator/test7.json
@@ -0,0 +1,13 @@
+{
+    "test": "Test 7",
+    "arguments": {
+        "--totalMessages":  2223,
+        "--start":  "",
+        "--testFile": "test7.json",
+        "--testDir": "test7",
+        "--timberWinRConfig": "test7-tw.json",
+        "--numMessages": 1234,
+        "--logLevel": "debug",       
+        "--resultsFile": "results7.json"
+    }
+}
--- a/TimberWinR.UnitTests/JsonFilterTests.cs
+++ b/TimberWinR.UnitTests/JsonFilterTests.cs
@@ -77,5 +77,56 @@ namespace TimberWinR.UnitTests
            Assert.IsNull(nostuff);
            Assert.AreEqual(6, jsonInputLine2.Count);            
        }
+
+        [Test]
+        public void TestRenameAndAdds()
+        {
+            JObject jsonInputLine1 = new JObject
+            {               
+                {"type", "Win32-FileLog"},
+                {"ComputerName", "dev.mycompany.net"},
+                {"Text", "{\"log4net:Username\" : \"NT AUTHORITY\",\"Email\":\"james@example.com\",\"Active\":true,\"CreatedDate\":\"2013-01-20T00:00:00Z\",\"Roles\":[\"User\",\"Admin\"]}"}
+            };
+
+
+            string jsonFilter = @"{  
+            ""TimberWinR"":{        
+                ""Filters"":[  
+	                {  
+		            ""json"":{  
+		                ""type"": ""Win32-FileLog"",		               
+                        ""source"": ""Text"",
+                        ""promote"": ""Text"",
+                        ""add_field"":[  
+                            ""test1"",
+                            ""value1"",
+                            ""test2"",
+                            ""value2""
+                        ],       
+                        ""rename"":[  
+                            ""Active"",
+                            ""active"",
+                            ""log4net:Username"",
+                            ""lusername""
+                        ]          
+		            }
+	              }]
+                }
+            }";
+            
+
+            // Positive Tests
+            Configuration c = Configuration.FromString(jsonFilter);
+            Json jf = c.Filters.First() as Json;
+            Assert.IsTrue(jf.Apply(jsonInputLine1));
+
+            Assert.AreEqual("NT AUTHORITY", jsonInputLine1["lusername"].ToString());
+            Assert.AreEqual("True", jsonInputLine1["active"].ToString());
+            Assert.IsNotNull(jsonInputLine1["test1"]);
+            Assert.IsNotNull(jsonInputLine1["test2"]);
+            Assert.AreEqual("value1", jsonInputLine1["test1"].ToString());
+            Assert.AreEqual("value2", jsonInputLine1["test2"].ToString());
+        }
+ 
    }
 }
--- a/TimberWinR.UnitTests/Parser/ElasticsearchOutputTests.cs
+++ b/TimberWinR.UnitTests/Parser/ElasticsearchOutputTests.cs
@@ -1,4 +1,6 @@
-namespace TimberWinR.UnitTests.Parser
+using TimberWinR.Outputs;
+
+namespace TimberWinR.UnitTests.Parser
 {
    using System;

@@ -51,5 +53,51 @@

            Assert.AreEqual("someindex-" + DateTime.UtcNow.ToString("yyyy.MM.dd"), result);
        }
+
+        [Test]
+        public void Given_no_ssl_then_validate_does_not_throw()
+        {
+            parser.Ssl = false;
+            Assert.That(() => parser.Validate(), Throws.Nothing);
+        }
+
+        [Test]
+        public void Given_ssl_and_no_username_then_validate_throws()
+        {
+            parser.Ssl = true;
+            parser.Password = "pass";
+
+            Assert.That(() => parser.Validate(), Throws.Exception.InstanceOf<ElasticsearchOutputParameters.ElasticsearchBasicAuthException>());
+        }
+
+        [Test]
+        public void Given_ssl_and_no_password_then_validate_throws()
+        {
+            parser.Ssl = true;
+            parser.Username = "user";
+
+            Assert.That(() => parser.Validate(), Throws.Exception.InstanceOf<ElasticsearchOutputParameters.ElasticsearchBasicAuthException>());
+        }
+
+        [Test]
+        public void Given_ssl_and_username_and_password_then_validate_does_not_throw()
+        {
+            parser.Ssl = true;
+            parser.Username = "user";
+            parser.Password = "pass";
+
+            Assert.That(() => parser.Validate(), Throws.Nothing);
+        }
+
+        [Test]
+        [TestCase("host", 1234, false, null, null, "http://host:1234/")]
+        [TestCase("host", 1234, true, "user", "pass", "https://user:pass@host:1234/")]
+        [TestCase("host", 1234, true, "user:", "pass@", "https://user%3A:pass%40@host:1234/")]
+        public void ComposeUri_Matches_Expected(string host, int port, bool ssl, string username, string password, string expectedUri)
+        {
+            var uri = ElasticsearchOutput.ComposeUri(host, port, ssl, username, password);
+
+            Assert.That(uri.ToString(), Is.EqualTo(expectedUri));
+        }
    }
 }
--- a/TimberWinR.sln
+++ b/TimberWinR.sln
@@ -116,4 +116,7 @@ Global
 	GlobalSection(SolutionProperties) = preSolution
 		HideSolutionNode = FALSE
 	EndGlobalSection
+	GlobalSection(Performance) = preSolution
+		HasPerformanceSessions = true
+	EndGlobalSection
 EndGlobal
--- a/TimberWinR/Configuration.cs
+++ b/TimberWinR/Configuration.cs
@@ -39,6 +39,12 @@ namespace TimberWinR
            get { return _events; }
        }

+        private List<StatsDOutputParameters> _statsdOutputs = new List<StatsDOutputParameters>();
+        public IEnumerable<StatsDOutputParameters> StatsDOutputs
+        {
+            get { return _statsdOutputs; }
+        }
+
        private List<RedisOutputParameters> _redisOutputs = new List<RedisOutputParameters>();
        public IEnumerable<RedisOutputParameters> RedisOutputs
        {
@@ -58,6 +64,12 @@ namespace TimberWinR
            get { return _stdoutOutputs; }
        }

+        private List<FileOutputParameters> _fileOutputs = new List<FileOutputParameters>();
+        public IEnumerable<FileOutputParameters> FileOutputs
+        {
+            get { return _fileOutputs; }
+        }
+
        private List<TcpParameters> _tcps = new List<TcpParameters>();
        public IEnumerable<TcpParameters> Tcps
        {
@@ -259,12 +271,16 @@ namespace TimberWinR

            if (x.TimberWinR.Outputs != null)
            {
+                if (x.TimberWinR.Outputs.StatsD != null)
+                    c._statsdOutputs.AddRange(x.TimberWinR.Outputs.StatsD.ToList());
                if (x.TimberWinR.Outputs.Redis != null)
                    c._redisOutputs.AddRange(x.TimberWinR.Outputs.Redis.ToList());
                if (x.TimberWinR.Outputs.Elasticsearch != null)
                    c._elasticsearchOutputs.AddRange(x.TimberWinR.Outputs.Elasticsearch.ToList());
                if (x.TimberWinR.Outputs.Stdout != null)
                    c._stdoutOutputs.AddRange(x.TimberWinR.Outputs.Stdout.ToList());
+                if (x.TimberWinR.Outputs.File != null)
+                    c._fileOutputs.AddRange(x.TimberWinR.Outputs.File.ToList());
            }

            if (x.TimberWinR.Filters != null)
@@ -299,9 +315,11 @@ namespace TimberWinR
            _events = new List<WindowsEvent>();
            _iisw3clogs = new List<IISW3CLogParameters>();
            _logs = new List<LogParameters>();
+            _statsdOutputs = new List<StatsDOutputParameters>();
            _redisOutputs = new List<RedisOutputParameters>();
            _elasticsearchOutputs = new List<ElasticsearchOutputParameters>();
            _stdoutOutputs = new List<StdoutOutputParameters>();
+            _fileOutputs = new List<FileOutputParameters>();
            _tcps = new List<TcpParameters>();
            _udps = new List<UdpParameters>();
        }
--- a/TimberWinR/Filters/GrokFilter.cs
+++ b/TimberWinR/Filters/GrokFilter.cs
@@ -30,8 +30,6 @@ namespace TimberWinR.Parser
        }
    }

-
-
    public partial class Grok : LogstashFilter
    {
        public override JObject ToJson()
@@ -92,35 +90,37 @@ namespace TimberWinR.Parser
            return true;
        }

+        // Test for any true matching condition(s)
        private bool Matches(Newtonsoft.Json.Linq.JObject json)
        {
-            string field = Match[0];
-            string expr = Match[1];
-
-            JToken token = null;
-            if (json.TryGetValue(field, out token))
+            for (int i = 0; i < Match.Length; i += 2)
            {
-                string text = token.ToString();
-                if (!string.IsNullOrEmpty(text))
+                string field = Match[i];
+                string expr = Match[i + 1];
+
+                JToken token = null;
+                if (json.TryGetValue(field, out token))
                {
-                    var resolver = new RegexGrokResolver();
-                    var pattern = resolver.ResolveToRegex(expr);
-                    var match = Regex.Match(text, pattern);
-                    if (match.Success)
+                    string text = token.ToString();
+                    if (!string.IsNullOrEmpty(text))
                    {
-                        var regex = new Regex(pattern);
-                        var namedCaptures = regex.MatchNamedCaptures(text);
-                        foreach (string fieldName in namedCaptures.Keys)
+                        var resolver = new RegexGrokResolver();
+                        var pattern = resolver.ResolveToRegex(expr);
+                        var match = Regex.Match(text, pattern);
+                        if (match.Success)
                        {
-                            AddOrModify(json, fieldName, namedCaptures[fieldName]);
+                            var regex = new Regex(pattern);
+                            var namedCaptures = regex.MatchNamedCaptures(text);
+                            foreach (string fieldName in namedCaptures.Keys)
+                            {
+                                AddOrModify(json, fieldName, namedCaptures[fieldName]);
+                            }
+                            return true; // Yes!
                        }
-                        return true; // Yes!
                    }
+                    if (string.IsNullOrEmpty(expr))
+                        return true; // Empty field is no match                 
                }
-                if (string.IsNullOrEmpty(expr))
-                    return true; // Empty field is no match
-                else
-                    return false;
            }
            return false; // Not specified is failure
        }
@@ -136,7 +136,7 @@ namespace TimberWinR.Parser
                    AddOrModify(json, fieldName, fieldValue);
                }
            }
-        }
+        }       

        private void RemoveFields(Newtonsoft.Json.Linq.JObject json)
        {
--- a/TimberWinR/Filters/JsonFilter.cs
+++ b/TimberWinR/Filters/JsonFilter.cs
@@ -92,9 +92,12 @@ namespace TimberWinR.Parser

                    if (Rename != null && Rename.Length > 0)
                    {
-                        string oldName = ExpandField(Rename[0], json);
-                        string newName = ExpandField(Rename[1], json);
-                        RenameProperty(json, oldName, newName);
+                        for (int i = 0; i < Rename.Length; i += 2)
+                        {
+                            string oldName = ExpandField(Rename[i], json);
+                            string newName = ExpandField(Rename[i+1], json);
+                            RenameProperty(json, oldName, newName);
+                        }
                    }

                    if (RemoveSource)
--- a/TimberWinR/Inputs/InputListener.cs
+++ b/TimberWinR/Inputs/InputListener.cs
@@ -44,6 +44,11 @@ namespace TimberWinR.Inputs
                             .ToString();    
        }

+        public void SetTypeName(string newType)
+        {
+            _typeName = newType;
+        }
+
        public bool HaveSeenFile(string fileName)
        {
            return Files.Contains(fileName);
@@ -96,6 +101,34 @@ namespace TimberWinR.Inputs
                LogManager.GetCurrentClassLogger().Error(ex);
            }          
        }
+       
+        protected void AddOrModify(JObject json, string fieldName, string fieldValue)
+        {
+            if (json[fieldName] == null)
+                json.Add(fieldName, fieldValue);
+            else
+                json[fieldName] = fieldValue;
+        }
+
+        protected void RenameProperty(JObject json, string oldName, string newName)
+        {
+            JToken token = json[oldName];
+            if (token != null)
+            {
+                json.Add(newName, token.DeepClone());
+                json.Remove(oldName);
+            }
+        }
+
+        protected string ExpandField(string fieldName, JObject json)
+        {
+            foreach (var token in json.Children().ToList())
+            {
+                string replaceString = "%{" + token.Path + "}";
+                fieldName = fieldName.Replace(replaceString, json[token.Path].ToString());
+            }
+            return fieldName;
+        }

        protected void EnsureRollingCaught()
        {
@@ -123,6 +156,7 @@ namespace TimberWinR.Inputs
            }           
        }

+
        public virtual void AddDefaultFields(JObject json)
        {
            if (json["type"] == null)
--- a/TimberWinR/Inputs/LogsFileDatabase.cs
+++ b/TimberWinR/Inputs/LogsFileDatabase.cs
@@ -30,10 +30,18 @@ namespace TimberWinR.Inputs

        private static LogsFileDatabase instance;

+        private bool ExistingFile(string logName)
+        {
+            lock (_locker)
+            {
+                return ExistingFileTest(logName);
+            }
+        }
+       
        //
        // Lookup the database entry for this log file, returns null if there isnt one.
        //
-        private LogsFileDatabaseEntry FindFileWithLock(string logName)
+        private LogsFileDatabaseEntry FindFile(string logName)
        {
            lock (_locker)
            {
@@ -61,12 +69,12 @@ namespace TimberWinR.Inputs
            }
        }

-        private LogsFileDatabaseEntry AddFileEntryWithLock(string logName)
+        private LogsFileDatabaseEntry AddFileEntry(string logName)
        {
            var de = new LogsFileDatabaseEntry();
            lock (_locker)
            {
-                var fi = new FileInfo(logName);
+                var fi = new FileInfo(logName);             
                de.FileName = logName;
                de.LogFileExists = fi.Exists;
                de.Previous = "";
@@ -74,21 +82,21 @@ namespace TimberWinR.Inputs
                de.ProcessedFile = false;
                de.LastPosition = fi.Length;
                de.SampleTime = DateTime.UtcNow;
-                de.CreationTimeUtc = fi.CreationTimeUtc;
+                de.CreationTimeUtc = fi.CreationTimeUtc;               

                Entries.Add(de);
                WriteDatabaseFileNoLock();
            }
            return de;
-        }
+        }      

        public static LogsFileDatabaseEntry LookupLogFile(string logName)
        {
-            LogsFileDatabaseEntry dbe = Instance.FindFileWithLock(logName);
+            LogsFileDatabaseEntry dbe = Instance.FindFile(logName);
            if (dbe == null)
-                dbe = Instance.AddFileEntryWithLock(logName);
+                dbe = Instance.AddFileEntry(logName);

-            FileInfo fi = new FileInfo(logName);
+            FileInfo fi = new FileInfo(logName);          

            dbe.LogFileExists = fi.Exists;
            var creationTime = fi.CreationTimeUtc;
@@ -99,18 +107,18 @@ namespace TimberWinR.Inputs
                dbe.Previous = "";
            }
            dbe.CreationTimeUtc = creationTime;
-
+      
            return dbe;
        }

        // Find all the non-existent entries and remove them.
-        private void PruneFilesWithLock()
+        private void PruneFiles()
        {
            lock (_locker)
            {
-                foreach (var entry in Entries.ToList())
+                foreach(var entry in Entries.ToList())
                {
-                    var fi = new FileInfo(entry.FileName);
+                    FileInfo fi = new FileInfo(entry.FileName);
                    if (!fi.Exists)
                        Entries.Remove(entry);
                }
@@ -122,7 +130,7 @@ namespace TimberWinR.Inputs
        {
            dbe.ProcessedFile = processedFile;
            dbe.LogFileExists = File.Exists(dbe.FileName);
-            Instance.UpdateEntryWithLock(dbe, lastOffset);
+            Instance.UpdateEntry(dbe, lastOffset);           
        }

        public static void Roll(LogsFileDatabaseEntry dbe)
@@ -130,20 +138,20 @@ namespace TimberWinR.Inputs
            dbe.ProcessedFile = false;
            dbe.LastPosition = 0;
            dbe.Previous = "";
-            Instance.UpdateEntryWithLock(dbe, 0);
-            dbe.NewFile = true;
+            Instance.UpdateEntry(dbe, 0);
+            dbe.NewFile = true;           
        }

-        private void UpdateEntryWithLock(LogsFileDatabaseEntry dbe, long lastOffset)
+        private void UpdateEntry(LogsFileDatabaseEntry dbe, long lastOffset)
        {
            lock (_locker)
            {
-                var fi = new FileInfo(dbe.FileName);
+                var fi = new FileInfo(dbe.FileName);                
                dbe.NewFile = !fi.Exists;
                dbe.CreationTimeUtc = fi.CreationTimeUtc;
-                dbe.SampleTime = DateTime.UtcNow;
+                dbe.SampleTime = DateTime.UtcNow;                 
                dbe.LastPosition = lastOffset;
-
+              
                WriteDatabaseFileNoLock();
            }
        }
@@ -167,7 +175,7 @@ namespace TimberWinR.Inputs
                        if (instance.Entries == null)
                            instance.Entries = new List<LogsFileDatabaseEntry>();

-                        instance.PruneFilesWithLock();
+                        instance.PruneFiles();
                    }
                }
                return instance;
@@ -187,17 +195,18 @@ namespace TimberWinR.Inputs
            }
            catch (Exception ex)
            {
-                LogManager.GetCurrentClassLogger().Error("Error reading database '{0}': {1}", DatabaseFileName, ex.ToString());
+                LogManager.GetCurrentClassLogger()
+                    .Error("Error reading database '{0}': {1}", DatabaseFileName, ex.ToString());
                try
                {
                    if (File.Exists(DatabaseFileName))
                        File.Delete(DatabaseFileName);
-                    LogManager.GetCurrentClassLogger().Error("Creating New Database '{0}'", DatabaseFileName);
+                    LogManager.GetCurrentClassLogger().Info("Creating New Database '{0}'", DatabaseFileName);
                    WriteDatabaseLock();
                }
                catch (Exception ex2)
                {
-                    LogManager.GetCurrentClassLogger().Error("Error Creating New Database '{0}': {1}", DatabaseFileName, ex2.ToString());
+                    LogManager.GetCurrentClassLogger().Info("Error Creating New Database '{0}': {1}", DatabaseFileName, ex2.ToString());
                }
            }
        }
@@ -249,7 +258,7 @@ namespace TimberWinR.Inputs
        public bool NewFile { get; set; }
        public bool ProcessedFile { get; set; }
        public bool LogFileExists { get; set; }
-        public string FileName { get; set; }
+        public string FileName { get; set; }       
        public DateTime CreationTimeUtc { get; set; }
        public DateTime SampleTime { get; set; }
        public long LastPosition { get; set; }
--- a/TimberWinR/Inputs/LogsListener.cs
+++ b/TimberWinR/Inputs/LogsListener.cs
@@ -53,6 +53,9 @@ namespace TimberWinR.Inputs
            if (_codecArguments != null && _codecArguments.Type == CodecArguments.CodecType.multiline)
                _codec = new Multiline(_codecArguments);

+            if (!string.IsNullOrEmpty(arguments.Type))
+                SetTypeName(arguments.Type);
+
            _receivedMessages = 0;
            _arguments = arguments;
            _pollingIntervalInSeconds = arguments.Interval;
--- a/TimberWinR/Inputs/TailFileListener.cs
+++ b/TimberWinR/Inputs/TailFileListener.cs
@@ -48,6 +48,9 @@ namespace TimberWinR.Inputs
            if (_codecArguments != null && _codecArguments.Type == CodecArguments.CodecType.multiline)
                _codec = new Multiline(_codecArguments);

+            if (!string.IsNullOrEmpty(arguments.Type))
+                SetTypeName(arguments.Type);
+
            _receivedMessages = 0;
            _arguments = arguments;
            _pollingIntervalInSeconds = arguments.Interval;
@@ -110,7 +113,7 @@ namespace TimberWinR.Inputs
            const int bufSize = 16535;
            long prevLen = offset;

-            var fi = new FileInfo(fileName);
+            FileInfo fi = new FileInfo(fileName);
            if (!fi.Exists)
                return;

@@ -120,9 +123,9 @@ namespace TimberWinR.Inputs
            {
                stream.Seek(prevLen, SeekOrigin.Begin);

-                var buffer = new char[bufSize];
-                var current = new StringBuilder();
-                using (var sr = new StreamReader(stream))
+                char[] buffer = new char[bufSize];
+                StringBuilder current = new StringBuilder();
+                using (StreamReader sr = new StreamReader(stream))
                {
                    int nRead;
                    do
--- a/TimberWinR/Inputs/TcpInputListener.cs
+++ b/TimberWinR/Inputs/TcpInputListener.cs
@@ -16,7 +16,7 @@ namespace TimberWinR.Inputs
        private Thread _listenThreadV4;
        private Thread _listenThreadV6;
        private readonly int _port;
-       
+        private TimberWinR.Parser.TcpParameters _arguments;
        private long _receivedMessages;
        private long _errorCount;

@@ -32,15 +32,19 @@ namespace TimberWinR.Inputs
            return json;
        }

-        public TcpInputListener(CancellationToken cancelToken, int port = 5140)
+        public TcpInputListener(TimberWinR.Parser.TcpParameters arguments, CancellationToken cancelToken, int port = 5140)
            : base(cancelToken, "Win32-Tcp")
        {
            _port = port;
+            _arguments = arguments;

            LogManager.GetCurrentClassLogger().Info("Tcp Input(v4/v6) on Port {0} Ready", _port);

-            _receivedMessages = 0;
+            if (!string.IsNullOrEmpty(arguments.Type))
+                SetTypeName(arguments.Type);

+            _receivedMessages = 0;
+           
            _tcpListenerV6 = new System.Net.Sockets.TcpListener(IPAddress.IPv6Any, port);
            _tcpListenerV4 = new System.Net.Sockets.TcpListener(IPAddress.Any, port);

@@ -91,6 +95,33 @@ namespace TimberWinR.Inputs
            }
        }

+        //
+        // Renames, and AddFields
+        //
+        private void ApplyFilters(JObject json)
+        {
+            if (_arguments.Renames != null)
+            {
+                for (int i = 0; i < _arguments.Renames.Length; i += 2)
+                {
+                    var oldName = ExpandField(_arguments.Renames[i], json);
+                    var newName = ExpandField(_arguments.Renames[i + 1], json);
+                    RenameProperty(json, oldName, newName);
+                }
+            }
+
+            if (_arguments.AddFields != null)
+            {
+                for (int i = 0; i < _arguments.AddFields.Length; i += 2)
+                {
+                    var fieldName = ExpandField(_arguments.AddFields[i], json);
+                    var fieldValue = ExpandField(_arguments.AddFields[i + 1], json);
+                    AddOrModify(json, fieldName, fieldValue);
+                }
+            }
+        }
+
+
        private void HandleNewClient(object client)
        {
            var tcpClient = (TcpClient)client;
@@ -109,6 +140,7 @@ namespace TimberWinR.Inputs
                            try
                            {
                                JObject json = JObject.Load(reader);
+                                ApplyFilters(json);
                                ProcessJson(json);
                                Interlocked.Increment(ref _receivedMessages);
                            }
--- a/TimberWinR/Inputs/UdpInputListener.cs
+++ b/TimberWinR/Inputs/UdpInputListener.cs
@@ -20,6 +20,7 @@ namespace TimberWinR.Inputs
        private long _parseErrors;
        private long _receiveErrors;
        private long _parsedMessages;
+        private TimberWinR.Parser.UdpParameters _arguments;

        public override JObject ToJson()
        {
@@ -35,10 +36,14 @@ namespace TimberWinR.Inputs
            return json;
        }

-        public UdpInputListener(CancellationToken cancelToken, int port = 5140) : base(cancelToken, "Win32-Udp")
+        public UdpInputListener(TimberWinR.Parser.UdpParameters arguments, CancellationToken cancelToken, int port = 5140) : base(cancelToken, "Win32-Udp")
        {
            _port = port;
            _receivedMessages = 0;
+            _arguments = arguments;
+
+            if (!string.IsNullOrEmpty(arguments.Type))
+                SetTypeName(arguments.Type);

            // setup raw data processor
            _unprocessedRawData = new BlockingCollection<byte[]>();
@@ -141,6 +146,32 @@ namespace TimberWinR.Inputs
            Finished();
        }

+        //
+        // Renames, and AddFields
+        //
+        private void ApplyFilters(JObject json)
+        {      
+            if (_arguments.Renames != null)
+            {
+                for (int i=0; i<_arguments.Renames.Length; i += 2)
+                {
+                    var oldName = ExpandField(_arguments.Renames[i], json);
+                    var newName = ExpandField(_arguments.Renames[i + 1], json);
+                    RenameProperty(json, oldName, newName);
+                }
+            }
+
+            if (_arguments.AddFields != null)
+            {
+                for (int i = 0; i < _arguments.AddFields.Length; i += 2)
+                {
+                    var fieldName = ExpandField(_arguments.AddFields[i], json);
+                    var fieldValue = ExpandField(_arguments.AddFields[i + 1], json);
+                    AddOrModify(json, fieldName, fieldValue);
+                }
+            }
+        }
+
        private void ProcessData(byte[] bytes)
        {
            var data = Encoding.UTF8.GetString(bytes, 0, bytes.Length);
@@ -148,8 +179,8 @@ namespace TimberWinR.Inputs
            try
            {
                var json = JObject.Parse(data);
+                ApplyFilters(json);
                ProcessJson(json);
-
                _parsedMessages++;
            }
            catch (Exception ex)
--- a/TimberWinR/Manager.cs
+++ b/TimberWinR/Manager.cs
@@ -24,7 +24,7 @@ namespace TimberWinR
        public Configuration Config { get; set; }
        public List<OutputSender> Outputs { get; set; }       
        public List<InputListener> Listeners { get; set; }
-        public bool LiveMonitor { get; set; }
+        public bool LiveMonitor { get; set; }      

        public event Action<Configuration> OnConfigurationProcessed;

@@ -47,7 +47,7 @@ namespace TimberWinR


        public void Shutdown()
-        {
+        {          
            LogManager.GetCurrentClassLogger().Info("Shutting Down");

            foreach (InputListener listener in Listeners)
@@ -68,7 +68,7 @@ namespace TimberWinR
        }

        public Manager(string jsonConfigFile, string logLevel, string logfileDir, bool liveMonitor, CancellationToken cancelToken, bool processConfiguration = true)
-        {
+        {          
            LogsFileDatabase.Manager = this;

            StartedOn = DateTime.UtcNow;
@@ -149,7 +149,7 @@ namespace TimberWinR
            if (processConfiguration)
            {
                ProcessConfiguration(cancelToken, Config);
-            }
+            }         
        }

        public void Start(CancellationToken cancelToken)
@@ -161,10 +161,19 @@ namespace TimberWinR
        {
            // Read the Configuration file
            if (config != null)
-            {
+            {                
                if (OnConfigurationProcessed != null)
                    OnConfigurationProcessed(config);

+                if (config.StatsDOutputs != null)
+                {
+                    foreach (var ro in config.StatsDOutputs)
+                    {
+                        var output = new StatsDOutput(this, ro, cancelToken);
+                        Outputs.Add(output);
+                    }
+                }
+
                if (config.RedisOutputs != null)
                {
                    foreach (var ro in config.RedisOutputs)
@@ -190,6 +199,15 @@ namespace TimberWinR
                    }
                }

+                if (config.FileOutputs != null)
+                {
+                    foreach (var ro in config.FileOutputs)
+                    {
+                        var output = new FileOutput(this, ro, cancelToken);
+                        Outputs.Add(output);
+                    }
+                }
+
                foreach (Parser.IISW3CLogParameters iisw3cConfig in config.IISW3C)
                {
                    var elistner = new IISW3CInputListener(iisw3cConfig, cancelToken);
@@ -232,7 +250,7 @@ namespace TimberWinR

                foreach (var tcp in config.Tcps)
                {
-                    var elistner = new TcpInputListener(cancelToken, tcp.Port);
+                    var elistner = new TcpInputListener(tcp, cancelToken, tcp.Port);
                    Listeners.Add(elistner);
                    foreach (var output in Outputs)
                        output.Connect(elistner);
@@ -240,7 +258,7 @@ namespace TimberWinR

                foreach (var udp in config.Udps)
                {
-                    var elistner = new UdpInputListener(cancelToken, udp.Port);
+                    var elistner = new UdpInputListener(udp, cancelToken, udp.Port);
                    Listeners.Add(elistner);
                    foreach (var output in Outputs)
                        output.Connect(elistner);
@@ -285,7 +303,7 @@ namespace TimberWinR
                    json.Add(new JProperty("type", "Win32-TimberWinR"));
                    json.Add(new JProperty("host", computerName));
                    output.Startup(json);
-                }
+                }               
            }
        }

--- a/TimberWinR/Outputs/Elasticsearch.cs
+++ b/TimberWinR/Outputs/Elasticsearch.cs
@@ -29,6 +29,9 @@ namespace TimberWinR.Outputs
    {
        private TimberWinR.Manager _manager;
        private readonly int _port;
+        private readonly bool _ssl;
+        private readonly string _username;
+        private readonly string _password;
        private readonly int _interval;
        private readonly int _flushSize;
        private readonly int _idleFlushTimeSeconds;
@@ -57,8 +60,8 @@ namespace TimberWinR.Outputs
            var nodes = new List<Uri>();
            foreach (var host in _hosts)
            {
-                var url = string.Format("http://{0}:{1}", host, _port);
-                nodes.Add(new Uri(url));
+                var uri = ComposeUri(host, _port, _ssl, _username, _password);
+                nodes.Add(uri);
            }
            var pool = new StaticConnectionPool(nodes.ToArray());
            var settings = new ConnectionSettings(pool)
@@ -73,6 +76,13 @@ namespace TimberWinR.Outputs
            return client;
        }

+        public static Uri ComposeUri(string host, int port, bool ssl, string username, string password)
+        {
+            return ssl
+                ? new Uri(string.Format("https://{0}:{1}@{2}:{3}", Uri.EscapeDataString(username), Uri.EscapeDataString(password), host, port))
+                : new Uri(string.Format("http://{0}:{1}", host, port));
+        }
+
        public ElasticsearchOutput(TimberWinR.Manager manager, Parser.ElasticsearchOutputParameters parameters, CancellationToken cancelToken)
            : base(cancelToken, "Elasticsearch")
        {
@@ -86,6 +96,9 @@ namespace TimberWinR.Outputs
            _timeout = parameters.Timeout;
            _manager = manager;
            _port = parameters.Port;
+            _ssl = parameters.Ssl;
+            _username = parameters.Username;
+            _password = parameters.Password;
            _interval = parameters.Interval;
            _hosts = parameters.Host;          
            _jsonQueue = new List<JObject>();
@@ -111,6 +124,9 @@ namespace TimberWinR.Outputs
                        new JProperty("messages", _sentMessages),
                        new JProperty("queuedMessageCount", _jsonQueue.Count),
                        new JProperty("port", _port),
+                        new JProperty("ssl", _ssl),
+                        new JProperty("username", _username),
+                        new JProperty("password", _password),
                        new JProperty("flushSize", _flushSize),                                       
                        new JProperty("idleFlushTime", _idleFlushTimeSeconds),     
                        new JProperty("interval", _interval),
--- a/TimberWinR/Outputs/File.cs
+++ b/TimberWinR/Outputs/File.cs
@@ -0,0 +1,140 @@
+using System.IO;
+using System.Text;
+using Newtonsoft.Json;
+using Newtonsoft.Json.Linq;
+using NLog;
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using System.Threading;
+using System.Threading.Tasks;
+
+namespace TimberWinR.Outputs
+{
+    public class FileOutput : OutputSender
+    {
+        private TimberWinR.Manager _manager;
+        private readonly int _interval;
+        private readonly object _locker = new object();
+        private readonly List<JObject> _jsonQueue;
+        private long _sentMessages;     
+        private Parser.FileOutputParameters _arguments;
+        public bool Stop { get; set; }
+
+        public FileOutput(TimberWinR.Manager manager, Parser.FileOutputParameters arguments, CancellationToken cancelToken)
+            : base(cancelToken, "File")
+        {
+            _arguments = arguments;
+            _sentMessages = 0;
+            _manager = manager;
+            _interval = arguments.Interval;
+            _jsonQueue = new List<JObject>();
+
+            var elsThread = new Task(FileSender, cancelToken);
+            elsThread.Start();
+        }
+
+        public override JObject ToJson()
+        {
+            JObject json = new JObject(
+                new JProperty("file-output",
+                    new JObject(
+                        new JProperty("queuedMessageCount", _jsonQueue.Count),
+                        new JProperty("sentMessageCount", _sentMessages))));
+
+            return json;
+        }
+
+        // 
+        // Pull off messages from the Queue, batch them up and send them all across
+        // 
+        private void FileSender()
+        {
+           
+            using (var syncHandle = new ManualResetEventSlim())
+            {
+                var fi = new FileInfo(_arguments.FileName);
+                if (File.Exists(_arguments.FileName))
+                    File.Delete(_arguments.FileName);
+
+                LogManager.GetCurrentClassLogger().Info("File Output Sending To: {0}", fi.FullName);
+       
+                using (StreamWriter sw = File.AppendText(_arguments.FileName))
+                {
+                    // Execute the query
+                    while (!Stop)
+                    {
+                        if (!CancelToken.IsCancellationRequested)
+                        {
+                            try
+                            {
+                                JObject[] messages;
+                                lock (_locker)
+                                {
+                                    messages = _jsonQueue.Take(_jsonQueue.Count).ToArray();
+                                    _jsonQueue.RemoveRange(0, messages.Length);
+                                }
+
+                                if (messages.Length > 0)
+                                {
+                                    try
+                                    {
+                                        foreach (JObject obj in messages)
+                                        {                                           
+                                            sw.WriteLine(obj.ToString(_arguments.ToFormat()));                                        
+                                            _sentMessages++;
+                                        }
+                                    }
+                                    catch (Exception ex)
+                                    {
+                                        LogManager.GetCurrentClassLogger().Error(ex);
+                                    }
+                                }
+                                if (!Stop)
+                                    syncHandle.Wait(TimeSpan.FromMilliseconds(_interval), CancelToken);
+                            }
+                            catch (OperationCanceledException)
+                            {
+                                break;
+                            }
+                            catch (Exception)
+                            {
+                            }
+                        }
+                    }
+                }
+            }
+        }
+
+        protected override void MessageReceivedHandler(Newtonsoft.Json.Linq.JObject jsonMessage)
+        {
+            if (_manager.Config.Filters != null)
+            {
+                if (ApplyFilters(jsonMessage))
+                    return;
+            }
+
+            var message = jsonMessage.ToString();
+   
+            lock (_locker)
+            {
+                _jsonQueue.Add(jsonMessage);
+            }
+        }
+
+        private bool ApplyFilters(JObject json)
+        {
+            bool drop = false;
+
+            foreach (var filter in _manager.Config.Filters)
+            {
+                if (!filter.Apply(json))
+                    drop = true;
+            }
+
+            return drop;
+        }
+
+    }
+}
+
--- a/TimberWinR/Outputs/Redis.cs
+++ b/TimberWinR/Outputs/Redis.cs
@@ -79,9 +79,11 @@ namespace TimberWinR.Outputs
        // Sample the queue and adjust the batch count if needed (ramp up slowly)
        public int UpdateCurrentBatchCount(int queueSize, int currentBatchCount)
        {
-            if (currentBatchCount < _maxBatchCount && currentBatchCount < queueSize && AverageQueueDepth() > currentBatchCount)
+            var avgQueueDepth = AverageQueueDepth();
+        
+            if (currentBatchCount < _maxBatchCount && currentBatchCount < queueSize && avgQueueDepth > currentBatchCount)
            {
-                currentBatchCount += Math.Max(_maxBatchCount / _batchCount, 1);
+                currentBatchCount += Math.Max(avgQueueDepth / _batchCount, _batchCount / 5);
                if (currentBatchCount >= _maxBatchCount && !_warnedReachedMax)
                {
                    LogManager.GetCurrentClassLogger().Warn("Maximum Batch Count of {0} reached.", currentBatchCount);
@@ -314,6 +316,9 @@ namespace TimberWinR.Outputs
                            {
                                _batchCounter.SampleQueueDepth(_jsonQueue.Count);
                                // Re-compute current batch size
+
+                                LogManager.GetCurrentClassLogger().Trace("{0}: Average Queue Depth: {1}, Current Length: {2}", Thread.CurrentThread.ManagedThreadId, _batchCounter.AverageQueueDepth(), _jsonQueue.Count); 
+       
                                _currentBatchCount = _batchCounter.UpdateCurrentBatchCount(_jsonQueue.Count, _currentBatchCount);

                                messages = _jsonQueue.Take(_currentBatchCount).ToArray();
--- a/TimberWinR/Outputs/StatsD.cs
+++ b/TimberWinR/Outputs/StatsD.cs
@@ -0,0 +1,368 @@
+using System;
+using System.Collections.Generic;
+using System.Diagnostics.Eventing.Reader;
+using System.Linq;
+using System.Linq.Expressions;
+using System.Net.Sockets;
+using System.Text;
+using System.Threading;
+using CSRedis;
+using Nest;
+using Newtonsoft.Json;
+using Newtonsoft.Json.Linq;
+using NLog;
+using System.Threading.Tasks;
+using RapidRegex.Core;
+using System.Text.RegularExpressions;
+using System.Globalization;
+using StatsdClient;
+using TimberWinR.Parser;
+
+namespace TimberWinR.Outputs
+{   
+    public class StatsDOutput : OutputSender
+    {
+        public int QueueDepth
+        {
+            get { return _jsonQueue.Count; }
+        }
+
+        public long SentMessages
+        {
+            get { return _sentMessages; }
+        }
+
+       
+        private readonly int _port;
+        public string _host { get; set; }
+
+        private readonly int _interval;
+        private readonly object _locker = new object();
+        private readonly List<JObject> _jsonQueue;
+        private TimberWinR.Manager _manager;        
+        private long _sentMessages;
+        private long _errorCount;
+        private readonly int _maxQueueSize;
+        private readonly bool _queueOverflowDiscardOldest;
+        private readonly int _flushSize;
+        private readonly int _idleFlushTimeSeconds;
+        private readonly int _numThreads;
+        private Parser.StatsDOutputParameters _params;
+
+        public bool Stop { get; set; }
+       
+        public override JObject ToJson()
+        {
+            var json = new JObject(
+                new JProperty("statsd",
+                    new JObject(                       
+                        new JProperty("errors", _errorCount),                       
+                        new JProperty("sentMessageCount", _sentMessages),
+                        new JProperty("queuedMessageCount", _jsonQueue.Count),
+                        new JProperty("port", _port),
+                        new JProperty("threads", _numThreads),
+                        new JProperty("flushSize", _flushSize),
+                        new JProperty("idleFlushTime", _idleFlushTimeSeconds),  
+                        new JProperty("maxQueueSize", _maxQueueSize),
+                        new JProperty("overflowDiscardOldest", _queueOverflowDiscardOldest),
+                        new JProperty("interval", _interval),
+                        new JProperty("host", _host)
+                        )));
+                       
+            return json;
+        }
+
+        public StatsDOutput(TimberWinR.Manager manager, Parser.StatsDOutputParameters parameters, CancellationToken cancelToken)
+            : base(cancelToken, "StatsD")
+        {
+            _params = parameters;
+            _manager = manager;                     
+            _port = parameters.Port;
+            _host = parameters.Host;
+            _interval = parameters.Interval;
+            _flushSize = parameters.FlushSize;
+            _idleFlushTimeSeconds = parameters.IdleFlushTimeInSeconds;
+            _maxQueueSize = parameters.MaxQueueSize;
+            _queueOverflowDiscardOldest = parameters.QueueOverflowDiscardOldest;
+            _numThreads = parameters.NumThreads;
+            _jsonQueue = new List<JObject>();
+
+            var metricsConfig = new MetricsConfig
+            {
+                StatsdServerName = _host,
+                Prefix = parameters.Namespace,
+            };
+
+            StatsdClient.Metrics.Configure(metricsConfig);
+
+            for (int i = 0; i < _numThreads; i++)
+            {
+                Task.Factory.StartNew(StatsDSender, cancelToken, TaskCreationOptions.LongRunning, TaskScheduler.Current);
+            }
+
+        }
+
+        public override string ToString()
+        {
+            return string.Format("StatsD Host: {0} Port: {1}", _host, _port);
+        }
+
+        /// <summary>
+        /// Forward on Json message to Redis Logstash queue
+        /// </summary>
+        /// <param name="jsonMessage"></param>
+        protected override void MessageReceivedHandler(JObject jsonMessage)
+        {
+            if (_manager.Config.Filters != null)
+            {
+                if (ApplyFilters(jsonMessage))
+                    return;
+            }
+           
+
+            var message = jsonMessage.ToString();
+            LogManager.GetCurrentClassLogger().Trace(message);
+
+            lock (_locker)
+            {
+                if (_jsonQueue.Count >= _maxQueueSize)
+                {
+                    // If we've exceeded our queue size, and we're supposed to throw out the oldest objects first,
+                    // then remove as many as necessary to get us under our limit
+                    if (_queueOverflowDiscardOldest)
+                    {
+                        LogManager.GetCurrentClassLogger()
+                            .Warn("Overflow discarding oldest {0} messages", _jsonQueue.Count - _maxQueueSize + 1);
+
+                        _jsonQueue.RemoveRange(0, (_jsonQueue.Count - _maxQueueSize) + 1);
+                    }
+                    // Otherwise we're in a "discard newest" mode, and this is the newest message, so just ignore it
+                    else
+                    {
+                        LogManager.GetCurrentClassLogger()
+                            .Warn("Overflow discarding newest message: {0}", message);
+
+                        return;
+                    }
+                }
+
+                _jsonQueue.Add(jsonMessage);
+            }
+        }
+
+        private bool ApplyFilters(JObject json)
+        {
+            bool drop = false;
+            foreach (var filter in _manager.Config.Filters)
+            {
+                if (!filter.Apply(json))
+                {
+                    LogManager.GetCurrentClassLogger().Debug("{0}: Dropping: {1}", Thread.CurrentThread.ManagedThreadId, json.ToString());
+                    drop = true;
+                }
+            }
+            // Check for matching type (if defined).
+            if (!drop && !string.IsNullOrEmpty(_params.InputType) && json["type"] != null)
+            {
+                string msgType = json["type"].ToString();
+                if (!string.IsNullOrEmpty(msgType) && msgType != _params.InputType)
+                    return true;
+            }
+
+            return drop;
+        }
+
+        // Places messages back into the queue (for a future attempt)
+        private void interlockedInsert(List<JObject> messages)
+        {
+            lock (_locker)
+            {
+                Interlocked.Increment(ref _errorCount);
+                _jsonQueue.InsertRange(0, messages);
+                if (_jsonQueue.Count > _maxQueueSize)
+                {
+                    LogManager.GetCurrentClassLogger().Warn("Exceeded maximum queue depth");
+                }
+            }
+        }
+
+        // 
+        // Pull off messages from the Queue, batch them up and send them all across
+        // 
+        private void StatsDSender()
+        {
+            DateTime lastFlushTime = DateTime.MinValue;
+
+            using (var syncHandle = new ManualResetEventSlim())
+            {
+                // Execute the query
+                while (!Stop)
+                {
+                    if (!CancelToken.IsCancellationRequested)
+                    {
+                        try
+                        {
+                            int messageCount = 0;
+                            List<JObject> messages = new List<JObject>();
+
+                            // Lets get whats in the queue
+                            lock (_locker)
+                            {
+                                messageCount = _jsonQueue.Count;
+
+                                // Time to flush?                             
+                                if (messageCount >= _flushSize || (DateTime.UtcNow - lastFlushTime).Seconds >= _idleFlushTimeSeconds)
+                                {
+                                    messages = _jsonQueue.Take(messageCount).ToList();
+                                    _jsonQueue.RemoveRange(0, messageCount);
+                                    if (messages.Count > 0)
+                                        _manager.IncrementMessageCount(messages.Count);
+                                }
+                            }
+
+                            TransmitStats(messages);
+
+                            if (!Stop)
+                                syncHandle.Wait(TimeSpan.FromMilliseconds(_interval), CancelToken);
+                        }
+                        catch (OperationCanceledException)
+                        {
+                            break;
+                        }
+                        catch (ThreadAbortException)
+                        {
+                            break;
+                        }
+                        catch (Exception ex)
+                        {                           
+                            Interlocked.Increment(ref _errorCount);
+                            LogManager.GetCurrentClassLogger().Error(ex);
+                        }
+                    }
+                }
+            }
+        }
+
+        protected string ExpandField(string fieldName, JObject json)
+        {
+            foreach (var token in json.Children())
+            {
+                string replaceString = "%{" + token.Path + "}";
+                fieldName = fieldName.Replace(replaceString, json[token.Path].ToString());
+            }
+            return fieldName;
+        }
+
+        private string BuildMetricPath(string metric, JObject json)
+        {
+            return string.Format("{0}.{1}", ExpandField(_params.Sender, json), ExpandField(metric, json));
+        }
+       
+        private void TransmitStats(List<JObject> messages)
+        {
+            // We've got some to send.
+            if (messages.Count > 0)
+            {
+                do
+                {
+                    try
+                    {
+                        int numMessages = messages.Count;
+                        foreach (var m in messages)
+                        {
+                            SendMetrics(m);
+                        }
+                        messages.RemoveRange(0, numMessages);
+                        Interlocked.Add(ref _sentMessages, numMessages);  
+                    }
+                    catch (Exception ex)
+                    {
+                        LogManager.GetCurrentClassLogger().Error(ex);
+                       
+                        interlockedInsert(messages);  // Put the messages back into the queue
+                        break;
+                    }
+                } while (messages.Count > 0);
+            }
+        }
+
+        // Process all the metrics for this json
+        private void SendMetrics(JObject m)
+        {
+            if (_params.Gauges != null && _params.Gauges.Length > 0)
+                DoGauges(m);
+            if (_params.Counts != null && _params.Counts.Length > 0)
+                DoCounts(m);
+            if (_params.Timings != null && _params.Timings.Length > 0)
+                DoTimings(m);
+            if (_params.Increments != null && _params.Increments.Length > 0)
+                DoIncrements(m);
+            if (_params.Decrements != null && _params.Decrements.Length > 0)
+                DoDecrements(m);
+        }
+
+        // Process the Gauges
+        private void DoGauges(JObject json)
+        {
+            for (int i=0; i<_params.Gauges.Length; i += 2)
+            {
+                string metricPath = BuildMetricPath(_params.Gauges[i], json);
+                string gaugeName = ExpandField(_params.Gauges[i + 1], json);
+                int value;
+                if (int.TryParse(gaugeName, out value))
+                {
+                    Metrics.Gauge(metricPath, value);
+                }
+            }
+        }
+
+        // Process the Gauges
+        private void DoTimings(JObject json)
+        {
+            for (int i = 0; i < _params.Timings.Length; i += 2)
+            {
+                string metricPath = BuildMetricPath(_params.Timings[i], json);
+                string timingName = ExpandField(_params.Timings[i + 1], json);
+                int value;
+                if (int.TryParse(timingName, out value))
+                {
+                    Metrics.Timer(metricPath, value, _params.SampleRate);
+                }
+            }
+        }
+
+        // Process the Counts
+        private void DoCounts(JObject json)
+        {
+            for (int i = 0; i < _params.Counts.Length; i += 2)
+            {
+                string metricPath = BuildMetricPath(_params.Counts[i], json);
+                string countName = ExpandField(_params.Counts[i + 1], json);
+                int value;
+                if (int.TryParse(countName, out value))
+                {
+                   Metrics.Counter(metricPath, value, _params.SampleRate);
+                }
+            }
+        }
+        // Process the Increments
+        private void DoIncrements(JObject json)
+        {
+            foreach (var metric in _params.Increments)
+            {
+                string metricPath = BuildMetricPath(metric, json);
+                Metrics.Counter(metricPath, 1,_params.SampleRate);
+            }
+        }
+
+        // Process the Increments
+        private void DoDecrements(JObject json)
+        {
+            foreach (var metric in _params.Increments)
+            {
+                string metricPath = BuildMetricPath(metric, json);
+                Metrics.Counter(metricPath, -1, _params.SampleRate);
+            }
+        }
+    }
+}
--- a/TimberWinR/Outputs/Stdout.cs
+++ b/TimberWinR/Outputs/Stdout.cs
@@ -34,6 +34,7 @@ namespace TimberWinR.Outputs
            JObject json = new JObject(
                new JProperty("stdout",
                    new JObject(
+                        new JProperty("queuedMessageCount", _jsonQueue.Count),
                        new JProperty("sentMessageCount", _sentMessages))));

            return json;
@@ -67,7 +68,7 @@ namespace TimberWinR.Outputs
                                    foreach (JObject obj in messages)
                                    {
                                        Console.WriteLine(obj.ToString());
-                                        _sentMessages++;
+                                        Interlocked.Increment(ref _sentMessages);
                                    }
                                }
                                catch (Exception ex)
--- a/TimberWinR/Parser.cs
+++ b/TimberWinR/Parser.cs
@@ -12,6 +12,7 @@ using Microsoft.SqlServer.Server;
 using Newtonsoft.Json;
 using Newtonsoft.Json.Linq;
 using NLog;
+using NLog.Config;
 using TimberWinR.Outputs;
 using System.CodeDom.Compiler;

@@ -43,10 +44,8 @@ namespace TimberWinR.Parser
            JToken token = json[oldName];
            if (token != null)
            {
+                json.Add(newName, token.DeepClone());
                json.Remove(oldName);
-                JToken newToken = json[newName];
-                if (newToken == null)
-                    json.Add(newName, token);
            }
        }

@@ -269,7 +268,7 @@ namespace TimberWinR.Parser
        public string Type { get; set; }

        [JsonProperty(PropertyName = "codec")]
-        public CodecArguments CodecArguments { get; set; }       
+        public CodecArguments CodecArguments { get; set; }

        [JsonProperty(PropertyName = "message")]
        public string Message { get; set; }
@@ -281,7 +280,7 @@ namespace TimberWinR.Parser
        public int Rate { get; set; }

        public void Validate()
-        {                      
+        {
        }

        public GeneratorParameters()
@@ -333,10 +332,12 @@ namespace TimberWinR.Parser

    public class TailFileArguments : IValidateSchema
    {
+        [JsonProperty(PropertyName = "type")]
+        public string Type { get; set; }
        [JsonProperty(PropertyName = "location")]
-        public string Location { get; set; }       
+        public string Location { get; set; }
        [JsonProperty(PropertyName = "recurse")]
-        public int Recurse { get; set; }        
+        public int Recurse { get; set; }
        [JsonProperty(PropertyName = "fields")]
        public List<Field> Fields { get; set; }
        [JsonProperty(PropertyName = "interval")]
@@ -363,6 +364,8 @@ namespace TimberWinR.Parser

    public class LogParameters : IValidateSchema
    {
+        [JsonProperty(PropertyName = "type")]
+        public string Type { get; set; }
        [JsonProperty(PropertyName = "location")]
        public string Location { get; set; }
        [JsonProperty(PropertyName = "iCodepage")]
@@ -399,15 +402,21 @@ namespace TimberWinR.Parser
    {
        [JsonProperty(PropertyName = "port")]
        public int Port { get; set; }
+        [JsonProperty(PropertyName = "type")]
+        public string Type { get; set; }
+        [JsonProperty("add_field")]
+        public string[] AddFields { get; set; }
+        [JsonProperty("rename")]
+        public string[] Renames { get; set; }

        public TcpParameters()
        {
            Port = 5140;
+            Type = "Win32-Tcp";
        }

        public void Validate()
        {
-
        }
    }

@@ -416,15 +425,21 @@ namespace TimberWinR.Parser
    {
        [JsonProperty(PropertyName = "port")]
        public int Port { get; set; }
+        [JsonProperty(PropertyName = "type")]
+        public string Type { get; set; }
+        [JsonProperty("add_field")]
+        public string[] AddFields { get; set; }
+        [JsonProperty("rename")]
+        public string[] Renames { get; set; }

        public UdpParameters()
        {
            Port = 5142;
+            Type = "Win32-Udp";
        }

        public void Validate()
        {
-
        }
    }
    public class W3CLogParameters : IValidateSchema
@@ -528,8 +543,98 @@ namespace TimberWinR.Parser
        }
    }

-    public class ElasticsearchOutputParameters
+    public class StatsDOutputParameters : IValidateSchema
    {
+        public class StatsDGaugeHashException : Exception
+        {
+            public StatsDGaugeHashException()
+                : base("StatsD output 'gauge' must be an array of pairs.")
+            {
+            }
+        }
+
+        public class StatsDCountHashException : Exception
+        {
+            public StatsDCountHashException()
+                : base("StatsD output 'count' must be an array of pairs.")
+            {
+            }
+        }
+        [JsonProperty(PropertyName = "type")]
+        public string InputType { get; set; }
+        [JsonProperty(PropertyName = "sender")]
+        public string Sender { get; set; }
+        [JsonProperty(PropertyName = "namespace")]
+        public string Namespace { get; set; }    
+        [JsonProperty(PropertyName = "host")]
+        public string Host { get; set; }
+        [JsonProperty(PropertyName = "port")]
+        public int Port { get; set; }
+        [JsonProperty(PropertyName = "interval")]
+        public int Interval { get; set; }
+        [JsonProperty(PropertyName = "flush_size")]
+        public int FlushSize { get; set; }
+        [JsonProperty(PropertyName = "idle_flush_time")]
+        public int IdleFlushTimeInSeconds { get; set; }
+        [JsonProperty(PropertyName = "max_queue_size")]
+        public int MaxQueueSize { get; set; }
+        [JsonProperty(PropertyName = "queue_overflow_discard_oldest")]
+        public bool QueueOverflowDiscardOldest { get; set; }
+        [JsonProperty(PropertyName = "threads")]
+        public int NumThreads { get; set; }
+        [JsonProperty(PropertyName = "sample_rate")]
+        public double SampleRate { get; set; }
+        [JsonProperty(PropertyName = "increment")] // Array: metric names
+        public string[] Increments { get; set; }
+        [JsonProperty(PropertyName = "decrement")] // Array: metric names
+        public string[] Decrements { get; set; }
+        [JsonProperty(PropertyName = "gauge")] // Hash: metric_name => gauge
+        public string[] Gauges { get; set; }
+        [JsonProperty(PropertyName = "count")] // Hash: metric_name => count
+        public string[] Counts { get; set; }
+        [JsonProperty(PropertyName = "timing")] // Hash: metric_name => count
+        public string[] Timings { get; set; }
+
+        public StatsDOutputParameters()
+        {
+            SampleRate = 1;
+            Port = 8125;
+            Host = "localhost";
+            Interval = 5000;
+            FlushSize = 5000;
+            IdleFlushTimeInSeconds = 10;
+            QueueOverflowDiscardOldest = true;
+            MaxQueueSize = 50000;
+            NumThreads = 1;
+            Namespace = "timberwinr";
+            Sender = System.Environment.MachineName.ToLower() + "." +
+                     Microsoft.Win32.Registry.LocalMachine.OpenSubKey(
+                         @"SYSTEM\CurrentControlSet\services\Tcpip\Parameters")
+                         .GetValue("Domain", "")
+                         .ToString().ToLower();
+        }
+
+        public void Validate()
+        {
+            if (Gauges != null && Gauges.Length % 2 != 0)
+                throw new StatsDGaugeHashException();
+
+            if (Counts != null && Counts.Length % 2 != 0)
+                throw new StatsDCountHashException();
+        }
+    }
+
+
+    public class ElasticsearchOutputParameters : IValidateSchema
+    {
+        public class ElasticsearchBasicAuthException : Exception
+        {
+            public ElasticsearchBasicAuthException()
+                : base("Elasticsearch 'username' and 'password' properties must be set when SSL is enabled.")
+            {
+            }
+        }
+
        const string IndexDatePattern = "(%\\{(?<format>[^\\}]+)\\})";

        [JsonProperty(PropertyName = "host")]
@@ -538,6 +643,12 @@ namespace TimberWinR.Parser
        public string Index { get; set; }
        [JsonProperty(PropertyName = "port")]
        public int Port { get; set; }
+        [JsonProperty(PropertyName = "ssl")]
+        public bool Ssl { get; set; }
+        [JsonProperty(PropertyName = "username")]
+        public string Username { get; set; }
+        [JsonProperty(PropertyName = "password")]
+        public string Password { get; set; }
        [JsonProperty(PropertyName = "timeout")]
        public int Timeout { get; set; }
        [JsonProperty(PropertyName = "threads")]
@@ -553,9 +664,9 @@ namespace TimberWinR.Parser
        [JsonProperty(PropertyName = "max_queue_size")]
        public int MaxQueueSize { get; set; }
        [JsonProperty(PropertyName = "queue_overflow_discard_oldest")]
-        public bool QueueOverflowDiscardOldest { get; set; }        
+        public bool QueueOverflowDiscardOldest { get; set; }
        [JsonProperty(PropertyName = "enable_ping")]
-        public bool EnablePing { get; set; }      
+        public bool EnablePing { get; set; }
        [JsonProperty(PropertyName = "ping_timeout")]
        public int PingTimeout { get; set; }

@@ -565,6 +676,9 @@ namespace TimberWinR.Parser
            IdleFlushTimeInSeconds = 10;
            Protocol = "http";
            Port = 9200;
+            Ssl = false;
+            Username = string.Empty;
+            Password = string.Empty;
            Index = "";
            Host = new string[] { "localhost" };
            Timeout = 10000;
@@ -614,6 +728,11 @@ namespace TimberWinR.Parser
            return typeName;
        }

+        public void Validate()
+        {
+            if (Ssl && (string.IsNullOrWhiteSpace(Username) || string.IsNullOrWhiteSpace(Password)))
+                throw new ElasticsearchBasicAuthException();
+        }
    }

    public class RedisOutputParameters
@@ -646,7 +765,6 @@ namespace TimberWinR.Parser
            Host = new string[] { "localhost" };
            Timeout = 10000;
            BatchCount = 200;
-            MaxBatchCount = BatchCount*10;
            NumThreads = 1;
            Interval = 5000;
            QueueOverflowDiscardOldest = true;
@@ -654,6 +772,8 @@ namespace TimberWinR.Parser
        }
    }

+
+
    public class StdoutOutputParameters
    {
        [JsonProperty(PropertyName = "interval")]
@@ -665,6 +785,43 @@ namespace TimberWinR.Parser
        }
    }

+    public class FileOutputParameters
+    {
+        public enum FormatKind
+        {
+            none, indented
+        };
+
+        [JsonProperty(PropertyName = "interval")]
+        public int Interval { get; set; }
+
+        [JsonProperty(PropertyName = "file_name")]
+        public string FileName { get; set; }
+
+        [JsonProperty(PropertyName = "format")]
+        public FormatKind Format { get; set; }
+
+        public FileOutputParameters()
+        {
+            Format = FormatKind.none;
+            Interval = 1000;
+            FileName = "timberwinr.out";
+        }
+
+        public Newtonsoft.Json.Formatting ToFormat()
+        {
+            switch (Format)
+            {
+                case FormatKind.indented:
+                    return Newtonsoft.Json.Formatting.Indented;
+
+                case FormatKind.none:
+                default:
+                    return Newtonsoft.Json.Formatting.None;
+            }
+        }
+    }
+
    public class OutputTargets
    {
        [JsonProperty("Redis")]
@@ -675,6 +832,12 @@ namespace TimberWinR.Parser

        [JsonProperty("Stdout")]
        public StdoutOutputParameters[] Stdout { get; set; }
+
+        [JsonProperty("File")]
+        public FileOutputParameters[] File { get; set; }
+
+        [JsonProperty("StatsD")]
+        public StatsDOutputParameters[] StatsD { get; set; }
    }

    public class InputSources
@@ -751,7 +914,7 @@ namespace TimberWinR.Parser

        public override void Validate()
        {
-            if (Match == null || Match.Length != 2)
+            if (Match == null || Match.Length % 2 != 0)
                throw new GrokFilterException();

            if (AddTag != null && AddTag.Length % 2 != 0)
@@ -892,7 +1055,7 @@ namespace TimberWinR.Parser
        }
    }

-   
+
    public partial class Json : LogstashFilter
    {
        public class JsonMissingSourceException : Exception
@@ -976,7 +1139,7 @@ namespace TimberWinR.Parser

        [JsonProperty("grokFilters")]
        public Grok[] Groks { get; set; }
-      
+
        [JsonProperty("mutateFilters")]
        public Mutate[] Mutates { get; set; }

@@ -987,7 +1150,7 @@ namespace TimberWinR.Parser
        public Json[] Jsons { get; set; }

        [JsonProperty("geoipFilters")]
-        public GeoIP[] GeoIPs { get; set; }    
+        public GeoIP[] GeoIPs { get; set; }
    }

    public class TimberWinR
--- a/TimberWinR/ReleaseNotes.md
+++ b/TimberWinR/ReleaseNotes.md
@@ -3,6 +3,18 @@
 A Native Windows to Redis/Elasticsearch Logstash Agent which runs as a service.

 Version / Date
+### 1.3.26.0 - 2015-05-15
+1. Added StatsD outputter
+2. Fixed shutdown hang if shutdown was received before service was fully started up.
+3. Closed issue [#36](https://github.com/Cimpress-MCP/TimberWinR/issues/36)
+
+### 1.3.25.0 - 2015-04-30
+1. Fixed Issue [#49](https://github.com/Cimpress-MCP/TimberWinR/issues/49)
+2. Fixed potential non-thread safe when renaming properties
+3. Added add_field, rename support to Udp/Tcp Input Listeners
+4. Fixed issue with multiple renames (was previously only renaming the first one)
+5. Added File outputter for testing.
+
 ### 1.3.24.0 - 2015-04-29
 1. Fixed potential bug in TailFiles when tailing log files which are partially flushed
   to disk, it now will not process the line until the \r\n has been seen.
--- a/TimberWinR/TimberWinR.csproj
+++ b/TimberWinR/TimberWinR.csproj
@@ -60,13 +60,21 @@
    <Reference Include="NLog">
      <HintPath>..\packages\NLog.3.2.0.0\lib\net40\NLog.dll</HintPath>
    </Reference>
-    <Reference Include="RapidRegex.Core">
-      <HintPath>..\packages\RapidRegex.Core.1.0.0.2\lib\net40\RapidRegex.Core.dll</HintPath>
+    <Reference Include="RapidRegex.Core, Version=1.0.0.4, Culture=neutral, processorArchitecture=MSIL">
+      <SpecificVersion>False</SpecificVersion>
+      <HintPath>..\packages\RapidRegex.Core.1.0.0.4\lib\net40\RapidRegex.Core.dll</HintPath>
    </Reference>
    <Reference Include="RestSharp">
      <HintPath>..\packages\RestSharp.105.0.0\lib\net4\RestSharp.dll</HintPath>
    </Reference>
+    <Reference Include="StatsdClient">
+      <HintPath>..\packages\StatsdClient.1.0.0.19\lib\net35\StatsdClient.dll</HintPath>
+    </Reference>
+    <Reference Include="StatsdClient.Configuration">
+      <HintPath>..\packages\StatsdClient.1.0.0.19\lib\net35\StatsdClient.Configuration.dll</HintPath>
+    </Reference>
    <Reference Include="System" />
+    <Reference Include="System.configuration" />
    <Reference Include="System.Core" />
    <Reference Include="System.Linq.Dynamic">
      <HintPath>..\packages\System.Linq.Dynamic.1.0.4\lib\net40\System.Linq.Dynamic.dll</HintPath>
@@ -114,7 +122,9 @@
    <Compile Include="Manager.cs" />
    <Compile Include="Outputs\Elasticsearch.cs" />
    <Compile Include="Outputs\OutputSender.cs" />
+    <Compile Include="Outputs\StatsD.cs" />
    <Compile Include="Outputs\Redis.cs" />
+    <Compile Include="Outputs\File.cs" />
    <Compile Include="Outputs\Stdout.cs" />
    <Compile Include="Parser.cs" />
    <Compile Include="Properties\AssemblyInfo.cs" />
@@ -138,9 +148,11 @@
    </Content>
    <None Include="mdocs\Codec.md" />
    <None Include="mdocs\DateFilter.md" />
+    <None Include="mdocs\StatsD.md" />
    <None Include="mdocs\Filters.md" />
    <None Include="mdocs\GeoIPFilter.md" />
    <None Include="mdocs\Generator.md" />
+    <None Include="mdocs\FileOutput.md" />
    <None Include="mdocs\TailFiles.md" />
    <None Include="mdocs\UdpInput.md" />
    <None Include="mdocs\W3CInput.md" />
--- a/TimberWinR/mdocs/ElasticsearchOutput.md
+++ b/TimberWinR/mdocs/ElasticsearchOutput.md
@@ -3,7 +3,7 @@
 The Elasticsearch output passes on data directly to Elasticsearch.

 ## Parameters
-The following parameters are allowed when configuring the Redis output.
+The following parameters are allowed when configuring the Elasticsearch output.

 | Parameter     |   Type   |  Description                                                | Details               |  Default |
 | :-------------|:---------|:------------------------------------------------------------| :---------------------------  | :-- |
@@ -14,6 +14,9 @@ The following parameters are allowed when configuring the Redis output.
 | *interval*                      | integer  | Interval in milliseconds to sleep during batch sends        | Interval       | 5000 |
 | *max_queue_size*                | integer  | Maximum Elasticsearch queue depth       |  | 50000 |
 | *port*                          | integer  | Elasticsearch port number                                   | This port must be open  | 9200  |
+| *ssl*                           | bool  | If true, use an HTTPS connection to Elasticsearch. See [this page] (https://www.elastic.co/guide/en/found/current/elk-and-found.html#_using_logstash) for a configuration example. | *username* and *password* are also required for HTTPS connections. | false |
+| *username*                      | string | Username for Elasticsearch credentials. | Required for HTTPS connection. | |
+| *password*                      | string | Password for Elasticsearch credentials. | Required for HTTPS connection. | |
 | *queue_overflow_discard_oldest* | bool  | If true, discard oldest messages when max_queue_size reached otherwise discard newest |  | true |
 | *threads*                       | [string]    | Number of Threads                         | Number of worker threads processing messages | 1 |
 | *enable_ping* | bool  | If true, pings the server to test for keep alive |  | false |
--- a/TimberWinR/mdocs/FileOutput.md
+++ b/TimberWinR/mdocs/FileOutput.md
@@ -0,0 +1,27 @@
+# Output: File
+
+The File output passes on data into a text file.
+
+## Parameters
+The following parameters are allowed when configuring the File output.
+
+| Parameter     |   Type   |  Description                                                | Details               |  Default |
+| :-------------|:---------|:------------------------------------------------------------| :---------------------------  | :-- |
+| *interval*    | integer  | Interval in milliseconds to sleep before appending data       | Interval      | 1000 |
+| *file_name*   | string   | Name of the file to be created                              |               | timberwinr.out |
+
+Example Input:
+```json
+{
+    "TimberWinR": {
+        "Outputs": {
+            "File": [
+               { 
+                    "file_name":  "foo.out",   
+                    "interval": 1000                   
+                }
+            ]
+		}
+	}
+}
+```
--- a/TimberWinR/mdocs/GrokFilter.md
+++ b/TimberWinR/mdocs/GrokFilter.md
@@ -26,14 +26,14 @@ The following operations are allowed when mutating a field.

 | Operation       |     Type        | Description                                                            
 | :---------------|:----------------|:-----------------------------------------------------------------------|
-| *type*          | property:string |Type to which this filter applyes, if empty, applies to all types.
-| *condition*     | property:string |C# expression
-| *rename*        | property:array  |Rename one or more fields                                       
-| *match*         | property:string |Required field must match before any subsequent grok operations are executed.
 | *add_field*     | property:array  |If the filter is successful, add an arbitrary field to this event.  Field names can be dynamic and include parts of the event using the %{field} syntax.  This property must be specified in pairs.                                    
-| *remove_field*  | property:array  |If the filter is successful, remove arbitrary fields from this event.  Field names can be dynamic and include parts of the event using the %{field} syntax.                                
 | *add_tag*       | property:array  |If the filter is successful, add an arbitrary tag to this event.  Tag names can be dynamic and include parts of the event using the %{field} syntax.                                  
+| *condition*     | property:string |C# expression
+| *match*         | property:array  |Required field must match (any) before any subsequent grok operations are executed.
+| *remove_field*  | property:array  |If the filter is successful, remove arbitrary fields from this event.  Field names can be dynamic and include parts of the event using the %{field} syntax.                                
 | *remove_tag*    | property:array  |If the filter is successful, remove arbitrary tags from this event.  Field names can be dynamic and include parts of the event using the %{field} syntax.                          
+| *rename*        | property:array  |Rename one or more fields                                       
+| *type*          | property:string |Type to which this filter applies, if empty, applies to all types.

 ## Operation Details
 ### match 
@@ -67,6 +67,28 @@ Given this configuration
    }     
  ]
 ```
+
+Given this configuration
+```json
+  "Filters": [     
+    {
+	   "grok": {
+           "matches": [
+               "message",
+               "%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}"
+           ],
+           "add_tag": [               
+               "http_log"
+           ],
+           "add_field": [
+               "verb", "%{method}"
+           ]
+        }           
+    }     
+  ]
+```
+
+
 And if the message matches, then 6 fields would be added to the event:
 1. client=55.3.244.1
 2. method=GET
--- a/TimberWinR/mdocs/JsonFilter.md
+++ b/TimberWinR/mdocs/JsonFilter.md
@@ -7,25 +7,27 @@ The following operations are allowed when mutating a field.

 | Operation       |     Type        | Description                                                            
 | :---------------|:----------------|:-----------------------------------------------------------------------|
-| *type*          | property:string |Type to which this filter applies, if empty, applies to all types.
-| *condition*     | property:string |C# expression, if the expression is true, continue, otherwise, ignore
-| *remove_source* | property:bool   |If true, the source property is removed, default: true
-| *source*        | property:string |Required field indicates which field contains the Json to be parsed
-| *promote*       | property:string |If supplied any properties named *promote* will be promoted to top-level
-| *target*        | property:string |If suppled, the parsed json will be contained underneath a propery named *target*
 | *add_field*     | property:array  |If the filter is successful, add an arbitrary field to this event.  Field names can be dynamic and include parts of the event using the %{field} syntax.  This property must be specified in pairs.                                    
-| *remove_field*  | property:array  |If the filter is successful, remove arbitrary fields from this event.  Field names can be dynamic and include parts of the event using the %{field} syntax.                                
 | *add_tag*       | property:array  |If the filter is successful, add an arbitrary tag to this event.  Tag names can be dynamic and include parts of the event using the %{field} syntax.                                  
+| *condition*     | property:string |C# expression, if the expression is true, continue, otherwise, ignore
+| *promote*       | property:string |If supplied any properties named *promote* will be promoted to top-level
+| *remove_field*  | property:array  |If the filter is successful, remove arbitrary fields from this event.  Field names can be dynamic and include parts of the event using the %{field} syntax.                                
 | *remove_tag*    | property:array  |If the filter is successful, remove arbitrary tags from this event.  Field names can be dynamic and include parts of the event using the %{field} syntax.                          
+| *remove_source* | property:bool   |If true, the source property is removed, default: true
+| *rename*        | property:array  |Rename one or more fields                                       
+| *source*        | property:string |Required field indicates which field contains the Json to be parsed
+| *target*        | property:string |If suppled, the parsed json will be contained underneath a propery named *target*
+| *type*          | property:string |Type to which this filter applies, if empty, applies to all types.

 ## Operation Details
 ### source 
-The match field is required, the first argument is the field to inspect, and compare to the expression specified by the second
-argument.  In the below example, the message is spected to be something like this from a fictional sample log:
+The source field is required, and indicates what Field contains the target Json, In the
+below example, the [Logs](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/Logs.md) input produces a Text field, 
+which contains a line to be parsed as Json.

 Given this input configuration:

-Lets assume that a newline such as the following is appended to foo.jlog:
+Lets assume that a new line such as the following is appended to foo.jlog:, this would end up being the Text field
 ```
   {"Email":"james@example.com","Active":true,"CreatedDate":"2013-01-20T00:00:00Z","Roles":["User","Admin"]}
 ```
@@ -54,7 +56,7 @@ Lets assume that a newline such as the following is appended to foo.jlog:
        }       
 ```

-In the above example, the file foo.jlog is being tailed, and when a newline is appended, it is assumed
+In the above example, the file foo.jlog is being tailed, and when a new line is appended, it is assumed
 to be Json and is parsed from the Text field, the parsed Json is then inserted underneath a property *stuff*

 The resulting output would be:
@@ -84,8 +86,8 @@ The fields must be in pairs with oldname first and newname second.
        "target": "stuff",
        "source": "Text",
        "rename": [
-            "Text",
-            "Data"
+            "level",
+            "Level"
        ]                   
     } 
   }     
@@ -96,7 +98,9 @@ The fields must be in pairs with fieldName first and value second.
 ```json
  "Filters": [     
    {
-		"json": {      			
+		"json": {  
+          "type":  "Win32-FileLog",       
+          "source": "Text",   			
 			"add_field": [
              "ComputerName", "Host",
              "Username", "%{SID}"				         
@@ -112,7 +116,9 @@ Remove the fields.  More than one field can be specified at a time.
  "Filters": [     
    {
 		"json": {      			
-			"remove_tag": [             
+          "type":  "Win32-FileLog",       
+          "source": "Text",   			
+			"remove_field": [             
             "static_tag1",
             "Computer_%{Host}"
 			]
@@ -128,6 +134,8 @@ Adds the tag(s) to the tag array.
  "Filters": [     
    {
 		"json": {      			
+          "type":  "Win32-FileLog",       
+          "source": "Text",   			
 			"add_tag": [
               "foo_%{Host}",
 			   "static_tag1"      
@@ -143,6 +151,8 @@ Remove the tag(s) to the tag array.  More than one tag can be specified at a tim
  "Filters": [     
    {
 		"json": {      			
+          "type":  "Win32-FileLog",       
+          "source": "Text",   			
 			"remove_tag": [             
             "static_tag1",
             "Username"
--- a/TimberWinR/mdocs/Logs.md
+++ b/TimberWinR/mdocs/Logs.md
@@ -7,11 +7,12 @@ The following parameters are allowed when configuring WindowsEvents.

 | Parameter         |     Type       |  Description                                                             | Details               |  Default |
 | :---------------- |:---------------| :----------------------------------------------------------------------- | :---------------------------  | :-- |
+| *iCodepage*       | integer |Codepage of the text file.                                              | 0 is the system codepage, -1 is UNICODE.                         | 0  |
 | *location*        | string  |Location of file(s) to monitor                                           | Path to text file(s) including wildcards. |     |
 | *logSource*       | string  |Source name                                  | Used for conditions |     |
 | *recurse*         | integer |Max subdirectory recursion level.                                       | 0 disables subdirectory recursion; -1 enables unlimited recursion. | 0 |
 | *splitLongLines*  | boolean |Behavior when event messages or event category names cannot be resolved. |When a text line is longer than 128K characters, the format truncates the line and either discards the remaining of the line (when this parameter is set to "false"), or processes the remainder of the line as a new line (when this parameter is set to "true").| false |
-| *iCodepage*       | integer |Codepage of the text file.                                              | 0 is the system codepage, -1 is UNICODE.                         | 0  |
+| *type*            | string  |Typename for this Input                                         |  |  Win32-FileLog  |
 | [codec](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/Codec.md)  | object | Codec to use  |

 Example Input: Monitors all files (recursively) located at C:\Logs1\ matching *.log as a pattern.  I.e. C:\Logs1\foo.log, C:\Logs1\Subdir\Log2.log, etc.
@@ -39,3 +40,4 @@ After a successful parse of an event, the following fields are added:
 | LogFilename | STRING |Full path of the file containing this line | 
 | Index | INTEGER | Line number |
 | Text | STRING | Text line content  |
+| type | STRING | Win32-FileLog |
--- a/TimberWinR/mdocs/RedisOutput.md
+++ b/TimberWinR/mdocs/RedisOutput.md
@@ -7,15 +7,15 @@ The following parameters are allowed when configuring the Redis output.

 | Parameter     |   Type   |  Description                                                | Details               |  Default |
 | :-------------|:---------|:------------------------------------------------------------| :---------------------------  | :-- |
-| *threads*     | string   | Location of log files(s) to monitor                         | Number of worker theads to send messages | 1 |
 | *batch_count* | integer  | Sent as a single message                         | Number of messages to aggregate | 200 |
-| *max_batch_count* | integer  | Dynamically adjusted count maximum                      | Increases over time | batch_count*10 |
-| *interval*    | integer  | Interval in milliseconds to sleep during batch sends        | Interval      | 5000 |
+| *host*        | string | The hostname(s) of your Redis server(s) | IP or DNS name |  |
 | *index*       | string   | The name of the redis list                                  | logstash index name | logstash |
-| *host*        | [string] | The hostname(s) of your Redis server(s) | IP or DNS name |  |
-| *port*        | integer  | Redis port number                                           | This port must be open  | 6379  |
+| *interval*    | integer  | Interval in milliseconds to sleep during batch sends        | Interval      | 5000 |
+| *max_batch_count* | integer  | Dynamically adjusted count maximum                      | Increases over time | batch_count * 10 |
 | *max_queue_size* | integer  | Maximum redis queue depth       |  | 50000 |
+| *port*        | integer  | Redis port number                                           | This port must be open  | 6379  |
 | *queue_overflow_discard_oldest* | bool  | If true, discard oldest messages when max_queue_size reached otherwise discard newest |  | true |
+| *threads*     | string   | Location of log files(s) to monitor                         | Number of worker theads to send messages | 1 |

 Example Input:
 ```json
--- a/TimberWinR/mdocs/StatsD.md
+++ b/TimberWinR/mdocs/StatsD.md
@@ -0,0 +1,94 @@
+# Output: StatsD
+
+The StatsD output passes on data directly to StatsD. (https://github.com/etsy/statsd)
+
+## Parameters
+The following parameters are allowed when configuring the StatsD output.
+
+| Parameter     |   Type   |  Description                                                | Details               |  Default |
+| :-------------|:---------|:------------------------------------------------------------| :---------------------------  | :-- |
+| *count*                         | string | Array of (metric_name, gauge name) pairs counted | Must come in pairs  |  |
+| *decrement*                     | string | Array of metrics to be decremented |  |  |
+| *flush_size*                    | integer  | Maximum number of messages before flushing           |   | 50000  |
+| *gauge*                         | string | Array of (metric_name, gauge name) pairs gauged | Must come in pairs  |  |
+| *host*                          | string | Hostname or IP of StatsD server | localhost |  |
+| *idle_flush_time*               | integer  | Maximum number of seconds elapsed before triggering a flush           |   | 10  |
+| *increment*                     | string | Array of metrics to be incremented |  |  |
+| *interval*                      | integer  | Interval in milliseconds to sleep between sends     | Interval       | 5000 |
+| *max_queue_size*                | integer  | Maximum StatsD queue depth       |  | 50000 |
+| *namespace*                     | string | Namespace for stats | timberwinr |  |
+| *port*                          | integer  | StatsD port number                                   | This port must be open  | 8125  |
+| *queue_overflow_discard_oldest* | bool  | If true, discard oldest messages when max_queue_size reached otherwise discard newest |  | true |
+| *sample_rate*                   | integer  | StatsD sample rate           |   | 1  |
+| *sender*                        | string | Sender name | FQDN |  |
+| *threads*                       | string    | Number of Threads processing messages          | | 1 |
+| *timing*                        | string | Array of (metric_name, timing_name) pairs timed | Must come in pairs  |  |
+| *type*                          | string |Type to which this filter applies, if empty, applies to all types.
+
+### Example Usage
+Example Input: Tail an apache log file, and record counts for bytes and increments for response codes.
+
+sample-apache.log (snip)
+```
+180.76.5.25 - - [13/May/2015:17:02:26 -0700] "GET /frameset.htm HTTP/1.1" 404 89 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" "www.redlug.com"
+208.115.113.94 - - [13/May/2015:17:03:55 -0700] "GET /robots.txt HTTP/1.1" 200 37 "-" "Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com)" "redlug.com"
+208.115.113.94 - - [13/May/2015:17:03:55 -0700] "GET /robots.txt HTTP/1.1" 200 37 "-" "Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com)" "www.redlug.com"
+```
+
+Note: [COMBINEDAPACHELOG](https://github.com/elastic/logstash/blob/v1.4.2/patterns/grok-patterns) is a standard
+Grok Pattern.
+
+TimberWinR configuration
+
+```json
+{
+    "TimberWinR": {
+        "Inputs": {
+            "TailFiles": [
+                {
+                    "interval": 5,
+                    "logSource": "apache log files",
+                    "location": "..\\sample-apache.log",
+                    "recurse": -1
+                }
+            ]
+        },
+        "Filters": [
+            {
+                "grok": {
+                    "type": "Win32-TailLog",
+                    "match": [
+                        "Text",
+                        "%{COMBINEDAPACHELOG}"
+                    ]
+                }
+            }
+        ],
+        "Outputs": {            
+            "StatsD": [
+                {
+                    "type": "Win32-TailLog",
+                    "port": 8125,
+                    "host": "stats.mycompany.svc",
+                    "increment":  ["apache.response.%{response}"],
+                    "count":  ["apache.bytes", "%{bytes}"]
+                }
+            ]
+        }
+    }
+}
+
+```
+
+Assuming your FQDN is something like mymachine.mycompany.com, you should see the following in Graphite:
+
+```
+stats.counters.timberwinr.mymachine.mycompany.com.apache.bytes.count
+stats.counters.timberwinr.mymachine.mycompany.com.apache.bytes.rate
+stats.counters.timberwinr.mymachine.mycompany.com.apache.response.200.count
+stats.counters.timberwinr.mymachine.mycompany.com.apache.response.200.rate
+stats.counters.timberwinr.mymachine.mycompany.com.apache.response.404.count
+stats.counters.timberwinr.mymachine.mycompany.com.apache.response.404.rate
+...
+...
+```
--- a/TimberWinR/mdocs/TailFiles.md
+++ b/TimberWinR/mdocs/TailFiles.md
@@ -8,6 +8,7 @@ The following parameters are allowed when configuring WindowsEvents.

 | Parameter         |     Type       |  Description                                                             | Details               |  Default |
 | :---------------- |:---------------| :----------------------------------------------------------------------- | :---------------------------  | :-- |
+| *type*            | string  |Typename for this Input                                         |  |  Win32-TailLog  |
 | *location*        | string  |Location of file(s) to monitor                                           | Path to text file(s) including wildcards. |     |
 | *logSource*       | string  |Source name                                  | Used for conditions |     |
 | *recurse*         | integer |Max subdirectory recursion level.                                       | 0 disables subdirectory recursion; -1 enables unlimited recursion. | 0 |
@@ -39,3 +40,4 @@ After a successful parse of an event, the following fields are added:
 | LogFilename | STRING |Full path of the file containing this line | 
 | Index | INTEGER | Line number |
 | Text | STRING | Text line content  |
+| type | STRING | Win32-TailLog |
--- a/TimberWinR/mdocs/TcpInput.md
+++ b/TimberWinR/mdocs/TcpInput.md
@@ -5,9 +5,12 @@ The Tcp input will open a port and listen for properly formatted JSON and will f
 ## Parameters
 The following parameters are allowed when configuring the Tcp input.

-| Parameter         |     Type       |  Description                                                             | Details               |  Default |
-| :---------------- |:---------------| :----------------------------------------------------------------------- | :---------------------------  | :-- |
-| *port*        | integer  |Port number to open        | Must be an available port |     |
+| Parameter         |     Type         |  Description                                                             | Details               |  Default |
+| :---------------- |:-----------------| :----------------------------------------------------------------------- | :---------------------------  | :-- |
+| *add_field*       | property:array   |Add field(s) to this event.  Field names can be dynamic and include parts of the event using the %{field} syntax.  This property must be specified in pairs.  |     |
+| *port*            | integer          |Port number to open        | Must be an available port |     |
+| *rename*          | property:array   |Rename one or more fields  |  |  |                             
+| *type*            | string           |Typename for this Input    |  |  Win32-Tcp  |

 Example Input: Listen on Port 5140

--- a/TimberWinR/mdocs/UdpInput.md
+++ b/TimberWinR/mdocs/UdpInput.md
@@ -7,7 +7,10 @@ The following parameters are allowed when configuring the Udp input.

 | Parameter         |     Type       |  Description                                                             | Details               |  Default |
 | :---------------- |:---------------| :----------------------------------------------------------------------- | :---------------------------  | :-- |
-| *port*        | integer  |Port number to open        | Must be an available port |     |
+| *add_field*       | property:array   |Add field(s) to this event.  Field names can be dynamic and include parts of the event using the %{field} syntax.  This property must be specified in pairs.  |     |
+| *port*            | integer  |Port number to open        | Must be an available port |     |
+| *rename*          | property:array   |Rename one or more fields  |  |  |                             
+| *type*            | string   |Typename for this Input                                         |  |  Win32-Udp  |

 Example Input: Listen on Port 5142

--- a/TimberWinR/mdocs/WindowsEvents.md
+++ b/TimberWinR/mdocs/WindowsEvents.md
@@ -65,3 +65,5 @@ After a successful parse of an event, the following fields are added:
 | Message | STRING | The full event message  |
 | Data | STRING | The binary data associated with the event  |

+
+
--- a/TimberWinR/packages.config
+++ b/TimberWinR/packages.config
@@ -7,8 +7,9 @@
  <package id="NEST" version="1.3.1" targetFramework="net40" />
  <package id="Newtonsoft.Json" version="6.0.5" targetFramework="net40" />
  <package id="NLog" version="3.2.0.0" targetFramework="net40" />
-  <package id="RapidRegex.Core" version="1.0.0.2" targetFramework="net40" />
+  <package id="RapidRegex.Core" version="1.0.0.4" targetFramework="net40" />
  <package id="RestSharp" version="105.0.0" targetFramework="net40" />
+  <package id="StatsdClient" version="1.0.0.19" targetFramework="net40" />
  <package id="System.Linq.Dynamic" version="1.0.4" targetFramework="net40" />
  <package id="Topshelf" version="3.1.4" targetFramework="net40" />
 </packages>
--- a/TimberWix/Product.wxs
+++ b/TimberWix/Product.wxs
@@ -54,6 +54,8 @@
        <File Id="default.json" Source="$(var.TimberWinR.ServiceHost.TargetDir)\default.json"  />
        <File Id="Newtonsoft.Json.dll" Source="$(var.TimberWinR.ServiceHost.TargetDir)\Newtonsoft.Json.dll"  />
        <File Id="Nlog.dll" Source="$(var.TimberWinR.ServiceHost.TargetDir)\Nlog.dll"  />
+        <File Id="StatsdClient.dll" Source="$(var.TimberWinR.ServiceHost.TargetDir)\StatsdClient.dll"  />
+        <File Id="StatsdClient.Configuration.dll" Source="$(var.TimberWinR.ServiceHost.TargetDir)\StatsdClient.Configuration.dll"  />
        <File Id="RestSharp.dll" Source="$(var.TimberWinR.ServiceHost.TargetDir)\RestSharp.dll"  />
        <File Id="RapidRegex.Core.dll" Source="$(var.TimberWinR.ServiceHost.TargetDir)\RapidRegex.Core.dll"  />
        <File Id="TimberWinR.dll" Source="$(var.TimberWinR.ServiceHost.TargetDir)\TimberWinR.dll"  />
--- a/chocolateyUninstall.ps1.template
+++ b/chocolateyUninstall.ps1.template
@@ -2,6 +2,6 @@ $packageName = 'TimberWinR-${version}' # arbitrary name for the package, used in
 $installerType = 'msi' #only one of these: exe, msi, msu
 $scriptPath =  $(Split-Path $MyInvocation.MyCommand.Path)
 $fileFullPath = Join-Path $scriptPath 'TimberWinR-${version}.0.msi'
-$silentArgs = '{CC4DF908-07C4-4BD8-A9FA-6E6AC315E30B} /quiet' 
+$silentArgs = '{E001D138-669B-4604-88C5-02C756461C15} /quiet' 
 $validExitCodes = @(0) #please insert other valid exit codes here, exit codes for ms http://msdn.microsoft.com/en-us/library/aa368542(VS.85).aspx
 UnInstall-ChocolateyPackage "$packageName" "$installerType" "$silentArgs" "fileFullPath" -validExitCodes $validExitCodes
--- a/timberwinr.jpg
+++ b/timberwinr.jpg
Author	SHA1	Message	Date
Tommy Parnell	5165224c65	change DEPRECATION notice	2016-01-20 08:06:25 -05:00
Ryan Breen	fd9306e24e	Update README.md Add deprecation notice.	2015-11-05 14:53:57 -05:00
Greg Lutz	050ae7cf84	Merge pull request #59 from schmidt4brains/feature/AddHttpsBasicAuthSupport Added support for HTTPS authentication to Found clusters	2015-09-11 06:46:29 -04:00
Doug Schmidt	a99b04e1b1	Added support for HTTPS authentication to Found clusters Elasticsearch's newly acquired found.io hosted clusters require HTTPS basic authentication in order to feed the index. This feature branch adds optional support for HTTPS according to https://www.elastic.co/guide/en/found/current/elk-and-found.html#_using_logstash When SSL is true, a non-empty username and password are required to authenticate against an Elasticsearch cluster.	2015-08-31 17:15:36 -07:00
Greg Lutz	dddbf6ae0d	logo	2015-07-13 11:26:50 -04:00
Greg Lutz	147e243478	Update WindowsEvents.md	2015-06-19 11:47:45 -04:00
Eric Fontana	7cf1230d7a	Merge pull request #52 from Cimpress-MCP/add_statsd_output Added StatsD outputter and smart shutdown code.	2015-05-18 07:00:36 -04:00
Eric Fontana	4fe5cec9e2	More doc tweaks to explain where it will show up.	2015-05-15 11:17:20 -04:00
Eric Fontana	9308f91316	I just realized this closed issue #36	2015-05-15 11:06:21 -04:00
Eric Fontana	28d20199ab	Updated Release notes and bumped version number to 1.3.26.0	2015-05-15 11:05:06 -04:00
Eric Fontana	5692d2ec42	Switched to use Nuget component for StatsD instead of including the code.	2015-05-15 10:41:09 -04:00
Eric Fontana	4dbb926698	Remove internal names.	2015-05-15 09:57:11 -04:00
Eric Fontana	92f23c1117	Added StatsD outputter and smart shutdown code.	2015-05-15 09:53:24 -04:00
Eric Fontana	cd31cf47f5	Merge pull request #51 from Cimpress-MCP/issue49 Release candidate 1.3.25.0	2015-04-30 13:14:23 -04:00
Eric Fontana	8fe3b1815c	Incorporated review changes	2015-04-30 13:12:44 -04:00
Eric Fontana	58ae2db05c	Remove internal names.	2015-04-30 10:20:24 -04:00
Eric Fontana	f1c4343cf6	Fixed typo	2015-04-30 10:10:15 -04:00
Eric Fontana	3669acbb26	Release 1.3.25.0 candidate	2015-04-30 10:09:07 -04:00
Eric Fontana	6600bb2b85	Added type parameter for Log/Tail inputs	2015-04-29 10:58:13 -04:00
Eric Fontana	50473a1f5a	Merge pull request #50 from Cimpress-MCP/add_input_generator All updates for 1.3.24.0	2015-04-29 10:47:41 -04:00