choco build 2

Backing off plan for rel 2.0.0

README.md

@@ -2,14 +2,6 @@ TimberWinR
 ==========
 A Native Windows to Redis/Elasticsearch Logstash Agent which runs as a service.
 
-## Development and Roadmap
-
-TimberWinR's development has been and in the short term will continue to be on a hiatus. The project's ownership has been recently transferred, and thus the project will be slow to pick up steam again. Currently a roadmap is being established, and a vision for the project is currently being formulated.
-
-#### Getting Involved
-
-If you wish to be involved with TimberWinR in an on-going basis file an issue.
-
 ## Why have TimberWinR?
 TimberWinR is a native .NET implementation utilizing Microsoft's [LogParser](http://technet.microsoft.com/en-us/scriptcenter/dd919274.aspx).  This means
 no JVM/JRuby is required, and LogParser does all the heavy lifting.  TimberWinR collects
@@ -17,7 +9,7 @@ the data from LogParser and ships it to Logstash via Redis (or can ship direcly
 
 ## Release Notes
 
-[View Version History](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/ReleaseNotes.md)
+[View Version History](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/ReleaseNotes.md)
 
 ## Basics
 TimberWinR uses a configuration file to control how the logs are collected, filtered and shipped off.  
@@ -38,27 +30,27 @@ Latest Build:
 
 ## Inputs
 The current supported Input format sources are:
- 1. [Logs](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/Logs.md) (Files, a.k.a Tailing a file)
+ 1. [Logs](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/Logs.md) (Files, a.k.a Tailing a file)
- 2. [Tcp](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/TcpInput.md) (listens on TCP port for JSON messages)
+ 2. [Tcp](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/TcpInput.md) (listens on TCP port for JSON messages)
- 3. [IISW3C](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/IISW3CInput.md)(Internet Information Services W3C Format)
+ 3. [IISW3C](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/IISW3CInput.md)(Internet Information Services W3C Format)
- 4. [WindowsEvents](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/WindowsEvents.md) (Windows Event Viewer)
+ 4. [WindowsEvents](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/WindowsEvents.md) (Windows Event Viewer)
- 5. [Stdin](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/StdinInput.md) (Standard Input for Debugging)
+ 5. [Stdin](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/StdinInput.md) (Standard Input for Debugging)
- 6. [W3C](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/W3CInput.md)(Internet Information Services W3C Advanced/Custom Format)
+ 6. [W3C](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/W3CInput.md)(Internet Information Services W3C Advanced/Custom Format)
- 7. [Udp](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/UdpInput.md) (listens for UDP on port for JSON messages)
+ 7. [Udp](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/UdpInput.md) (listens for UDP on port for JSON messages)
- 8. [TailFiles](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/TailFiles.md) (Tails log files efficiently)
+ 8. [TailFiles](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/TailFiles.md) (Tails log files efficiently)
- 8. [Generator](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/Generator.md) (Generate logs for testing *New*)
+ 8. [Generator](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/Generator.md) (Generate logs for testing *New*)
 
 ## Codecs
 The current list of supported codecs are:
- 1. [Multiline](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/Codec.md)
+ 1. [Multiline](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/Codec.md)
 
 ## Filters
 The current list of supported filters are:
- 1. [Grok](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/GrokFilter.md)
+ 1. [Grok](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/GrokFilter.md)
- 2. [Mutate](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/MutateFilter.md)
+ 2. [Mutate](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/MutateFilter.md)
- 3. [Date](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/DateFilter.md)
+ 3. [Date](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/DateFilter.md)
- 4. [Json](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/JsonFilter.md)
+ 4. [Json](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/JsonFilter.md)
- 5. [GeoIP](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/GeoIPFilter.md)
+ 5. [GeoIP](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/GeoIPFilter.md)
 
 Note that there are now two syntaxes for filters, array and single, if you want more than one filter of the same
 type then you must use the array tag instead of the singular tag. i.e:
@@ -112,11 +104,11 @@ Since TimberWinR only ships to Redis and Elasticsearch, the format generated by
 represented as a JSON Property or Array.
 
 ## Outputs
-1. [Redis](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/RedisOutput.md)
+1. [Redis](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/RedisOutput.md)
-2. [Elasticsearch](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/ElasticsearchOutput.md)
+2. [Elasticsearch](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/ElasticsearchOutput.md)
-3. [Stdout](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/StdoutOutput.md)
+3. [Stdout](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/StdoutOutput.md)
-4. [File](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/FileOutput.md)
+4. [File](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/FileOutput.md)
-5. [StatsD](https://github.com/tparnell8/TimberWinR/blob/master/TimberWinR/mdocs/StatsD.md)
+5. [StatsD](https://github.com/Cimpress-MCP/TimberWinR/blob/master/TimberWinR/mdocs/StatsD.md)
 
 ## Sample Configuration
 TimberWinR reads a JSON configuration file, an example file is shown here:

TimberWinR.ServiceHost/Properties/AssemblyInfo.cs

@@ -32,5 +32,5 @@ using System.Runtime.InteropServices;
 // You can specify all the values or you can default the Build and Revision Numbers 
 // by using the '*' as shown below:
 // [assembly: AssemblyVersion("1.0.*")]
-[assembly: AssemblyVersion("1.3.26.0")]
+[assembly: AssemblyVersion("1.3.27.0")]
-[assembly: AssemblyFileVersion("1.3.26.0")]
+[assembly: AssemblyFileVersion("1.3.27.0")]

TimberWinR.UnitTests/Parser/ElasticsearchOutputTests.cs

@@ -1,6 +1,4 @@
-using TimberWinR.Outputs;
+namespace TimberWinR.UnitTests.Parser
-
-namespace TimberWinR.UnitTests.Parser
 {
     using System;
 
@@ -53,51 +51,5 @@ namespace TimberWinR.UnitTests.Parser
 
             Assert.AreEqual("someindex-" + DateTime.UtcNow.ToString("yyyy.MM.dd"), result);
         }
-
-        [Test]
-        public void Given_no_ssl_then_validate_does_not_throw()
-        {
-            parser.Ssl = false;
-            Assert.That(() => parser.Validate(), Throws.Nothing);
-        }
-
-        [Test]
-        public void Given_ssl_and_no_username_then_validate_throws()
-        {
-            parser.Ssl = true;
-            parser.Password = "pass";
-
-            Assert.That(() => parser.Validate(), Throws.Exception.InstanceOf<ElasticsearchOutputParameters.ElasticsearchBasicAuthException>());
-        }
-
-        [Test]
-        public void Given_ssl_and_no_password_then_validate_throws()
-        {
-            parser.Ssl = true;
-            parser.Username = "user";
-
-            Assert.That(() => parser.Validate(), Throws.Exception.InstanceOf<ElasticsearchOutputParameters.ElasticsearchBasicAuthException>());
-        }
-
-        [Test]
-        public void Given_ssl_and_username_and_password_then_validate_does_not_throw()
-        {
-            parser.Ssl = true;
-            parser.Username = "user";
-            parser.Password = "pass";
-
-            Assert.That(() => parser.Validate(), Throws.Nothing);
-        }
-
-        [Test]
-        [TestCase("host", 1234, false, null, null, "http://host:1234/")]
-        [TestCase("host", 1234, true, "user", "pass", "https://user:pass@host:1234/")]
-        [TestCase("host", 1234, true, "user:", "pass@", "https://user%3A:pass%40@host:1234/")]
-        public void ComposeUri_Matches_Expected(string host, int port, bool ssl, string username, string password, string expectedUri)
-        {
-            var uri = ElasticsearchOutput.ComposeUri(host, port, ssl, username, password);
-
-            Assert.That(uri.ToString(), Is.EqualTo(expectedUri));
-        }
     }
 }

TimberWinR/Diagnostics/Diagnostics.cs

@@ -8,7 +8,7 @@ using System.Threading;
 using System.Net;
 using System.Net.Sockets;
 using System.IO;
-
+using System.Linq.Expressions;
 using Newtonsoft.Json.Linq;
 
 using NLog;
@@ -52,29 +52,48 @@ namespace TimberWinR.Diagnostics
         public JObject DiagnosticsOutput()
         {
             JObject json = new JObject(
-                           new JProperty("timberwinr",
+                new JProperty("timberwinr",
-                               new JObject(
+                    new JObject(
-                                   new JProperty("version", Assembly.GetEntryAssembly().GetName().Version.ToString()),
+                        new JProperty("version", Assembly.GetEntryAssembly().GetName().Version.ToString()),
-                                   new JProperty("messages", Manager.NumMessages),
+                        new JProperty("messages", Manager.NumMessages),
-                                   new JProperty("startedon", Manager.StartedOn),
+                        new JProperty("startedon", Manager.StartedOn),
-                                   new JProperty("configfile", Manager.JsonConfig),
+                        new JProperty("configfile", Manager.JsonConfig),
-                                   new JProperty("logdir", Manager.LogfileDir),
+                        new JProperty("logdir", Manager.LogfileDir),
-                                   new JProperty("logginglevel", LogManager.GlobalThreshold.ToString()),
+                        new JProperty("logginglevel", LogManager.GlobalThreshold.ToString())
-                                   new JProperty("inputs",
+                        )));
-                                       new JArray(
+            AddDiagnosis(json);
-                                           from i in Manager.Listeners
-                                           select new JObject(i.ToJson()))),
-                                   new JProperty("filters",
-                                       new JArray(
-                                           from f in Manager.Config.Filters
-                                           select new JObject(f.ToJson()))),
-                                   new JProperty("outputs",
-                                       new JArray(
-                                           from o in Manager.Outputs
-                                           select new JObject(o.ToJson()))))));
             return json;
         }
 
+        protected void AddDiagnosis(JObject wrapper)
+        {
+            wrapper.Add("inputs", GetDiagnosisByType("inputs", Manager.Listeners.ToList<IDiagnosable>()));
+            wrapper.Add("filters", GetDiagnosisByType("filters", Manager.Config.Filters.ToList<IDiagnosable>()));
+            wrapper.Add("outputs", GetDiagnosisByType("inputs", Manager.Outputs.ToList<IDiagnosable>()));
+        }
+
+        protected JObject GetDiagnosisByType(String type, List<IDiagnosable> diags)
+        {
+            JObject category = new JObject();
+            foreach(IDiagnosable diag in diags)
+            {
+                JArray array = GetTypeArray(diag.GetType().Name.ToString(), category);
+                array.Add(diag.ToJson());
+            }
+            return category;
+        }
+
+        protected JArray GetTypeArray(String name, JObject category)
+        {
+            JArray ret = (JArray)category.GetValue(name);
+            if (ret == null)
+            {
+                ret = new JArray();
+                category.Add(new JProperty(name, ret));
+            }
+            return ret;
+        }
+
         private void DiagnosticCallback(IAsyncResult result)
         {
             if (web == null)

TimberWinR/Diagnostics/IDiagnosable.cs

@@ -0,0 +1,13 @@
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using System.Text;
+using Newtonsoft.Json.Linq;
+
+namespace TimberWinR.Diagnostics
+{
+    public interface IDiagnosable
+    {
+        JObject ToJson();
+    }
+}

TimberWinR/Inputs/IISW3CInputListener.cs

@@ -135,8 +135,6 @@ namespace TimberWinR.Inputs
                                 rs.close();
                                 GC.Collect();
                             }
-                            if (!Stop)
-                                syncHandle.Wait(TimeSpan.FromSeconds(_pollingIntervalInSeconds), CancelToken);
                         }
                         catch (OperationCanceledException)
                         {
@@ -145,7 +143,18 @@ namespace TimberWinR.Inputs
                         catch (Exception ex)
                         {
                             LogManager.GetCurrentClassLogger().Error(ex);
-                        }                       
+                        }
+                        finally
+                        {
+                            try
+                            {
+                                if (!Stop)
+                                    syncHandle.Wait(TimeSpan.FromSeconds(_pollingIntervalInSeconds), CancelToken);
+                            }
+                            catch (Exception)
+                            {
+                            }
+                        }
                     }
                 }
             }

TimberWinR/Inputs/InputListener.cs

@@ -8,10 +8,11 @@ using System.Linq;
 using System.Text;
 using System.Threading;
 using NLog;
+using TimberWinR.Diagnostics;
 
 namespace TimberWinR.Inputs
 {
-    public abstract class InputListener
+    public abstract class InputListener: IDiagnosable
     {
         public CancellationToken CancelToken { get; set; }
         public event Action<JObject> OnMessageRecieved;

TimberWinR/Inputs/WindowsEvtInputListener.cs

@@ -8,6 +8,7 @@ using TimberWinR.Parser;
 using LogQuery = Interop.MSUtil.LogQueryClassClass;
 using EventLogInputFormat = Interop.MSUtil.COMEventLogInputContextClassClass;
 using LogRecordSet = Interop.MSUtil.ILogRecordset;
+using System.IO;
 
 namespace TimberWinR.Inputs
 {
@@ -97,12 +98,13 @@ namespace TimberWinR.Inputs
                     // Execute the query
                     if (!CancelToken.IsCancellationRequested)
                     {
+                        var oLogQuery = new LogQuery();
                         try
-                        {                          
+                        {
-                            var oLogQuery = new LogQuery();
 
                             var qfiles = string.Format("SELECT Distinct [EventLog] FROM {0}", location);
                             var rsfiles = oLogQuery.Execute(qfiles, iFmt);
+
                             for (; !rsfiles.atEnd(); rsfiles.moveNext())
                             {
                                 var record = rsfiles.getRecord();
@@ -113,7 +115,7 @@ namespace TimberWinR.Inputs
                                         logName);
                                     var rcount = oLogQuery.Execute(qcount, iFmt);
                                     var qr = rcount.getRecord();
-                                    var lrn = (Int64)qr.getValueEx("MaxRecordNumber");
+                                    var lrn = (Int64) qr.getValueEx("MaxRecordNumber");
                                     logFileMaxRecords[logName] = lrn;
                                 }
                             }
@@ -137,12 +139,13 @@ namespace TimberWinR.Inputs
                                         object v = record.getValue(field.Name);
                                         if (field.Name == "Data")
                                             v = ToPrintable(v.ToString());
-                                        if ((field.Name == "TimeGenerated" || field.Name == "TimeWritten") && field.DataType == typeof (DateTime))
+                                        if ((field.Name == "TimeGenerated" || field.Name == "TimeWritten") &&
+                                            field.DataType == typeof (DateTime))
                                             v = ((DateTime) v).ToUniversalTime();
                                         json.Add(new JProperty(field.Name, v));
                                     }
 
-                                    var lrn = (Int64)record.getValueEx("RecordNumber");
+                                    var lrn = (Int64) record.getValueEx("RecordNumber");
                                     logFileMaxRecords[fileName] = lrn;
 
                                     ProcessJson(json);
@@ -163,6 +166,24 @@ namespace TimberWinR.Inputs
                         {
                             LogManager.GetCurrentClassLogger().Error(ex);
                         }
+                        finally
+                        {
+                            try
+                            {
+                                oLogQuery = null;
+                                // Sleep 
+                                if (!Stop)
+                                    syncHandle.Wait(TimeSpan.FromSeconds(_pollingIntervalInSeconds), CancelToken);
+                            }
+                            catch (OperationCanceledException)
+                            {
+                            }
+                            catch (Exception ex1)
+                            {
+                                LogManager.GetCurrentClassLogger().Warn(ex1);
+                            }
+                            
+                        }
                     }
                 }
                 Finished();

TimberWinR/Outputs/Elasticsearch.cs

@@ -29,9 +29,6 @@ namespace TimberWinR.Outputs
     {
         private TimberWinR.Manager _manager;
         private readonly int _port;
-        private readonly bool _ssl;
-        private readonly string _username;
-        private readonly string _password;
         private readonly int _interval;
         private readonly int _flushSize;
         private readonly int _idleFlushTimeSeconds;
@@ -60,8 +57,8 @@ namespace TimberWinR.Outputs
             var nodes = new List<Uri>();
             foreach (var host in _hosts)
             {
-                var uri = ComposeUri(host, _port, _ssl, _username, _password);
+                var url = string.Format("http://{0}:{1}", host, _port);
-                nodes.Add(uri);
+                nodes.Add(new Uri(url));
             }
             var pool = new StaticConnectionPool(nodes.ToArray());
             var settings = new ConnectionSettings(pool)
@@ -76,13 +73,6 @@ namespace TimberWinR.Outputs
             return client;
         }
 
-        public static Uri ComposeUri(string host, int port, bool ssl, string username, string password)
-        {
-            return ssl
-                ? new Uri(string.Format("https://{0}:{1}@{2}:{3}", Uri.EscapeDataString(username), Uri.EscapeDataString(password), host, port))
-                : new Uri(string.Format("http://{0}:{1}", host, port));
-        }
-
         public ElasticsearchOutput(TimberWinR.Manager manager, Parser.ElasticsearchOutputParameters parameters, CancellationToken cancelToken)
             : base(cancelToken, "Elasticsearch")
         {
@@ -96,9 +86,6 @@ namespace TimberWinR.Outputs
             _timeout = parameters.Timeout;
             _manager = manager;
             _port = parameters.Port;
-            _ssl = parameters.Ssl;
-            _username = parameters.Username;
-            _password = parameters.Password;
             _interval = parameters.Interval;
             _hosts = parameters.Host;          
             _jsonQueue = new List<JObject>();
@@ -124,9 +111,6 @@ namespace TimberWinR.Outputs
                         new JProperty("messages", _sentMessages),
                         new JProperty("queuedMessageCount", _jsonQueue.Count),
                         new JProperty("port", _port),
-                        new JProperty("ssl", _ssl),
-                        new JProperty("username", _username),
-                        new JProperty("password", _password),
                         new JProperty("flushSize", _flushSize),                                       
                         new JProperty("idleFlushTime", _idleFlushTimeSeconds),     
                         new JProperty("interval", _interval),

TimberWinR/Outputs/OutputSender.cs

@@ -4,11 +4,12 @@ using System.Linq;
 using System.Text;
 using System.Threading;
 using Newtonsoft.Json.Linq;
+using TimberWinR.Diagnostics;
 using TimberWinR.Inputs;
 
 namespace TimberWinR.Outputs
 {
-    public abstract class OutputSender
+    public abstract class OutputSender : IDiagnosable
     {
         public CancellationToken CancelToken { get; private set; }        
         private List<InputListener> _inputs;

TimberWinR/Outputs/Redis.cs

@@ -132,6 +132,7 @@ namespace TimberWinR.Outputs
         private long _errorCount;
         private long _redisDepth;
         private DateTime? _lastErrorTimeUTC;
+        private DateTime? _lastSentTimeUTC;
         private readonly int _maxQueueSize;
         private readonly bool _queueOverflowDiscardOldest;
         private BatchCounter _batchCounter;
@@ -180,6 +181,7 @@ namespace TimberWinR.Outputs
                         new JProperty("host", string.Join(",", _redisHosts)),
                         new JProperty("errors", _errorCount),
                         new JProperty("lastErrorTimeUTC", _lastErrorTimeUTC),
+                        new JProperty("lastSentTimeUTC", _lastSentTimeUTC),
                         new JProperty("redisQueueDepth", _redisDepth),
                         new JProperty("sentMessageCount", _sentMessages),
                         new JProperty("queuedMessageCount", _jsonQueue.Count),
@@ -317,9 +319,13 @@ namespace TimberWinR.Outputs
                                 _batchCounter.SampleQueueDepth(_jsonQueue.Count);
                                 // Re-compute current batch size
 
-                                LogManager.GetCurrentClassLogger().Trace("{0}: Average Queue Depth: {1}, Current Length: {2}", Thread.CurrentThread.ManagedThreadId, _batchCounter.AverageQueueDepth(), _jsonQueue.Count); 
+                                LogManager.GetCurrentClassLogger()
-       
+                                    .Trace("{0}: Average Queue Depth: {1}, Current Length: {2}",
-                                _currentBatchCount = _batchCounter.UpdateCurrentBatchCount(_jsonQueue.Count, _currentBatchCount);
+                                        Thread.CurrentThread.ManagedThreadId, _batchCounter.AverageQueueDepth(),
+                                        _jsonQueue.Count);
+
+                                _currentBatchCount = _batchCounter.UpdateCurrentBatchCount(_jsonQueue.Count,
+                                    _currentBatchCount);
 
                                 messages = _jsonQueue.Take(_currentBatchCount).ToArray();
                                 _jsonQueue.RemoveRange(0, messages.Length);
@@ -340,7 +346,9 @@ namespace TimberWinR.Outputs
                                             {
                                                 client.StartPipe();
                                                 LogManager.GetCurrentClassLogger()
-                                                    .Debug("{0}: Sending {1} Messages to {2}", Thread.CurrentThread.ManagedThreadId, messages.Length, client.Host);
+                                                    .Debug("{0}: Sending {1} Messages to {2}",
+                                                        Thread.CurrentThread.ManagedThreadId, messages.Length,
+                                                        client.Host);
 
                                                 try
                                                 {
@@ -348,6 +356,7 @@ namespace TimberWinR.Outputs
                                                     Interlocked.Add(ref _sentMessages, messages.Length);
                                                     client.EndPipe();
                                                     sentSuccessfully = true;
+                                                    _lastSentTimeUTC = DateTime.UtcNow;
                                                     if (messages.Length > 0)
                                                         _manager.IncrementMessageCount(messages.Length);
                                                 }
@@ -391,9 +400,7 @@ namespace TimberWinR.Outputs
                                         _jsonQueue.InsertRange(0, messages);
                                     }
                                 }
-                            }                          
+                            }
-                            if (!Stop)
-                                syncHandle.Wait(TimeSpan.FromMilliseconds(_interval), CancelToken);
                         }
                         catch (OperationCanceledException)
                         {
@@ -409,6 +416,17 @@ namespace TimberWinR.Outputs
                             Interlocked.Increment(ref _errorCount);
                             LogManager.GetCurrentClassLogger().Error(ex);
                         }
+                        finally
+                        {
+                            try
+                            {
+                                if (!Stop)
+                                    syncHandle.Wait(TimeSpan.FromMilliseconds(_interval), CancelToken);
+                            }
+                            catch (Exception)
+                            {
+                            }
+                        }
                     }
                 }
             }

TimberWinR/Parser.cs

@@ -15,6 +15,7 @@ using NLog;
 using NLog.Config;
 using TimberWinR.Outputs;
 using System.CodeDom.Compiler;
+using TimberWinR.Diagnostics;
 
 namespace TimberWinR.Parser
 {
@@ -26,7 +27,7 @@ namespace TimberWinR.Parser
     }
 
 
-    public abstract class LogstashFilter : IValidateSchema
+    public abstract class LogstashFilter : IValidateSchema, IDiagnosable
     {
         public abstract bool Apply(JObject json);
 
@@ -625,16 +626,8 @@ namespace TimberWinR.Parser
     }
 
 
-    public class ElasticsearchOutputParameters : IValidateSchema
+    public class ElasticsearchOutputParameters
     {
-        public class ElasticsearchBasicAuthException : Exception
-        {
-            public ElasticsearchBasicAuthException()
-                : base("Elasticsearch 'username' and 'password' properties must be set when SSL is enabled.")
-            {
-            }
-        }
-
         const string IndexDatePattern = "(%\\{(?<format>[^\\}]+)\\})";
 
         [JsonProperty(PropertyName = "host")]
@@ -643,12 +636,6 @@ namespace TimberWinR.Parser
         public string Index { get; set; }
         [JsonProperty(PropertyName = "port")]
         public int Port { get; set; }
-        [JsonProperty(PropertyName = "ssl")]
-        public bool Ssl { get; set; }
-        [JsonProperty(PropertyName = "username")]
-        public string Username { get; set; }
-        [JsonProperty(PropertyName = "password")]
-        public string Password { get; set; }
         [JsonProperty(PropertyName = "timeout")]
         public int Timeout { get; set; }
         [JsonProperty(PropertyName = "threads")]
@@ -676,9 +663,6 @@ namespace TimberWinR.Parser
             IdleFlushTimeInSeconds = 10;
             Protocol = "http";
             Port = 9200;
-            Ssl = false;
-            Username = string.Empty;
-            Password = string.Empty;
             Index = "";
             Host = new string[] { "localhost" };
             Timeout = 10000;
@@ -728,11 +712,6 @@ namespace TimberWinR.Parser
             return typeName;
         }
 
-        public void Validate()
-        {
-            if (Ssl && (string.IsNullOrWhiteSpace(Username) || string.IsNullOrWhiteSpace(Password)))
-                throw new ElasticsearchBasicAuthException();
-        }
     }
 
     public class RedisOutputParameters

TimberWinR/TimberWinR.csproj

@@ -96,6 +96,7 @@
     <Compile Include="Configuration.cs" />
     <Compile Include="ConfigurationErrors.cs" />
     <Compile Include="Diagnostics\Diagnostics.cs" />
+    <Compile Include="Diagnostics\IDiagnosable.cs" />
     <Compile Include="Filters\DateFilter.cs" />
     <Compile Include="Filters\FilterBase.cs" />
     <Compile Include="Filters\GrokFilter.cs" />

TimberWinR/mdocs/ElasticsearchOutput.md

@@ -14,9 +14,6 @@ The following parameters are allowed when configuring the Elasticsearch output.
 | *interval*                      | integer  | Interval in milliseconds to sleep during batch sends        | Interval       | 5000 |
 | *max_queue_size*                | integer  | Maximum Elasticsearch queue depth       |  | 50000 |
 | *port*                          | integer  | Elasticsearch port number                                   | This port must be open  | 9200  |
-| *ssl*                           | bool  | If true, use an HTTPS connection to Elasticsearch. See [this page] (https://www.elastic.co/guide/en/found/current/elk-and-found.html#_using_logstash) for a configuration example. | *username* and *password* are also required for HTTPS connections. | false |
-| *username*                      | string | Username for Elasticsearch credentials. | Required for HTTPS connection. | |
-| *password*                      | string | Password for Elasticsearch credentials. | Required for HTTPS connection. | |
 | *queue_overflow_discard_oldest* | bool  | If true, discard oldest messages when max_queue_size reached otherwise discard newest |  | true |
 | *threads*                       | [string]    | Number of Threads                         | Number of worker threads processing messages | 1 |
 | *enable_ping* | bool  | If true, pings the server to test for keep alive |  | false |

appveyor.yml

@@ -1,14 +1,50 @@
-version: 1.2.{build}
+version: build_number_{build}
-
+configuration: Release
-build:
+skip_tags: true
-  verbosity: minimal
+init:
-
+- ps: "$v = [regex]::match($env:APPVEYOR_REPO_BRANCH,'rel-(.*)').Groups[1].Value\nWrite-Host \"On branch $($env:APPVEYOR_REPO_BRANCH)\"\nIF($v) { \n   $env:VERSION_FROM_BRANCH = \"$($v).$($env:APPVEYOR_BUILD_NUMBER)\"\n} else {\n   $env:VERSION_FROM_BRANCH = \"0.0.0.$($env:APPVEYOR_BUILD_NUMBER)\"\n}\nWrite-Host \"Set version to $($env:VERSION_FROM_BRANCH)\""
 assembly_info:
   patch: true
-  file: AssemblyInfo.*
+  file: '**\AssemblyInfo.*'
-  assembly_version: "1.2.{build}"
+  assembly_version: $(VERSION_FROM_BRANCH)
-  assembly_file_version: "{version}"
+  assembly_file_version: $(VERSION_FROM_BRANCH)
-  assembly_informational_version: "{version}"
+  assembly_informational_version: $(VERSION_FROM_BRANCH)
+before_build:
+- ps: >-
+    mkdir tools
 
+    NuGet.exe restore TimberWinR.sln
+build:
+  verbosity: normal
+after_build:
+- ps: >-
+    cat chocolateyInstall.ps1.template|%{$_-replace "\$\{version\}",$env:VERSION_FROM_BRANCH} > tools\chocolateyInstall.ps1
+
+    cat chocolateyUninstall.ps1.template|%{$_-replace "\$\{version\}",$env:VERSION_FROM_BRANCH} > tools\chocolateyUninstall.ps1
+
+    cat tools\chocolateyUninstall.ps1
+
+    cat timberwinr.nuspec.template|%{$_-replace "\$\{version\}",$env:VERSION_FROM_BRANCH} > timberwinr.nuspec
+
+    choco pack timberwinr.nuspec
 artifacts:
-  - path: '**\*.msi'
+- path: .\*.nupkg
+  name: NuGet
+- path: TimberWinR.ServiceHost\bin\*\*.dll
+  name: Dlls
+- path: TimberWinR.ServiceHost\bin\*\*.exe
+  name: Exes
+- path: TimberWix\bin\*\*.msi
+  name: MSI 
+deploy:
+- provider: NuGet
+  server: https://chocolatey.org/
+  skip_symbols: true
+  artifact: NuGet
+  on:
+    branch: /release/.*/
+- provider: GitHub
+  release: $(VERSION_FROM_BRANCH)
+  artifact: Dlls, Exes, MSI
+  on:
+    branch: /release/.*/