The server issue that occurred (discussed in the previous blog post) has been resolved.  The issue was linked to an anti-virus email scanner.  While the issue is not fully understood as it has not been duplicated on another system yet, it was unmistakably linked to this as disabling this scanner resulted in a total recovery of the system without having to restart services (note that prior, even restarting the services completely failed to resolve it).  The TCPView utility from SysInternals reported this scanner having a large number TCP endpoints (and possibly half-open or fully open connections) which were not shutting down.  It may have also overloaded the Named Pipes causing other havoc, but primarily the connections and endpoints appear to have caused the system to hit a limit preventing new ones from being created, essentially blocking the web server from communicating with the database.  The scanner has been shut down until the situation is understood better thus returning the server to a fully functioning state.  This scanner had been enabled and tested many weeks ago and was thought to be completely stable which made it difficult to zero in on the problem.  Why it suddently started behaving in this manner is unknown as there was not a seriously increased influx of spam or other messages that would clearly have led to this, and even so, it should have been designed to handle it better.

Thanks to our customers for their patience and prompt reporting of issues which was helpful in resolving the situation and we apologize for the interruptions that were caused.