I remember being on a project some 9 years ago and having to build one of these. To build a realtime gateway is not as easy as you would think. In my project there were accountants uploading invoices of various types and formats that we had to translate into text using an OCR software package. We built a workflow using a TIBCO workflow designer solution (which I wouldn’t hesitate now to replace with WF!)
At a certain point people from outside the organisation had the ability to upload a file and this file had to be intercepted by a gateway before being persisted and operated on the through the workflow. You would think that this was an easy and common solution to implement. However, at the time it wasn’t. We used a Symantec gateway product and its C API which allowed us to use the ICAP protocol and thus do real time scanning.
Begin everything with a web role
For the last 6 months I’ve wanted to talk about Microsoft Endpoint Protection (http://www.microsoft.com/en-us/download/details.aspx?id=29209) which is still in CTP as a I write this. It’s a lesser known plugin which exists for Windows Azure. For anybody that receives uploaded content, this should be a commonplace part of the design. In this piece I want to look at a pattern for rolling your gateway with Endpoint Protection. It’s not ideal because it literally is a virus scanner, enabling real time protection and certain other aspects but uses Diagnostics to show issues that have taken place.
The files which are part of the endpoint protection plugin
So initially we’ll enable the imports:
<Imports> <Import moduleName="Diagnostics" /> <Import moduleName="Antimalware" /> <Import moduleName="RemoteAccess" /> <Import moduleName="RemoteForwarder" /> </Imports>
You can see the addition of Antimalware here.
Correspondingly, our service configuration gives us the following new settings:
<Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="<my connection string>" /> <Setting name="Microsoft.WindowsAzure.Plugins.Antimalware.ServiceLocation" value="North Europe" /> <Setting name="Microsoft.WindowsAzure.Plugins.Antimalware.EnableAntimalware" value="true" /> <Setting name="Microsoft.WindowsAzure.Plugins.Antimalware.EnableRealtimeProtection" value="true" /> <Setting name="Microsoft.WindowsAzure.Plugins.Antimalware.EnableWeeklyScheduledScans" value="false" /> <Setting name="Microsoft.WindowsAzure.Plugins.Antimalware.DayForWeeklyScheduledScans" value="7" /> <Setting name="Microsoft.WindowsAzure.Plugins.Antimalware.TimeForWeeklyScheduledScans" value="120" /> <Setting name="Microsoft.WindowsAzure.Plugins.Antimalware.ExcludedExtensions" value="txt|rtf|jpg" /> <Setting name="Microsoft.WindowsAzure.Plugins.Antimalware.ExcludedPaths" value="" /> <Setting name="Microsoft.WindowsAzure.Plugins.Antimalware.ExcludedProcesses" value="" />
The settings are using Endpoint Protection for real time protection and scheduled scan. It’s obviously highly configurable like most virus scanners and in the background will update all malware defitions securely from a Microsoft source.
Endpoint protection installed on our webrole
First thing we’ll do is download a free virus test file from http://www.eicar.org/85-0-Download.html. Eicar has ensured that this definition is picked by most of the common virus scanning so Endpoint Protection should recognise this immediately. I’ve tested this with the .zip file but any of them are fine.
The first port of call is setting up diagnostics to proliferate the event log entries. We can do this within our RoleEntryPoint.OnStart method for our web role.
var config = DiagnosticMonitor.GetDefaultInitialConfiguration();
//exclude informational and verbose event log entries
config.WindowsEventLog.DataSources.Add("System!*[System[Provider[@Name='Microsoft Antimalware'] and (Level=1 or Level=2 or Level=3 or Level=4)]]");
//write to persisted storage every 1 minute
config.WindowsEventLog.ScheduledTransferPeriod = System.TimeSpan.FromMinutes(1.0);
DiagnosticMonitor.Start("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString", config);
Diagnostics info in Azure Management Studio
Okay, so in testing it looks like the whole process of cutting and pasting the file onto the desktop or another location takes about 10 seconds for the Endpoint Protection to pick this up and quarante the file. Given this we’ll set the bar at 20 seconds.
Endpoint protection discovers malware
I created a very simple ASP.NET web forms application with a file upload control. There are two ways to detect whether the file has been flagged as malware:
- Check to see whether the file is still around or has been removed and placed in quarantine
- Check the eventlog entry to see whether this has been flagged as malware.
We’re going to focus on No.2 so I’ve created a simple button click event which will persist the file. Endpoint protection will kick in within the short period so we’ll write the file to disk and then pause for 20 seconds. After our wait we’ll then check the eventlog and in the message string we’ll have a wealth of information about the file which has been quarantined.
bool hasFile = fuEndpointProtection.HasFile;
string path = "";
if(hasFile)
{
path = Path.Combine(Server.MapPath("."), fuEndpointProtection.FileName);
fuEndpointProtection.SaveAs(path);
}
// block here until we check endpoint protection to see whether the file has been delivered okay!
Thread.Sleep(20000);
var log = new EventLog("System", Environment.MachineName, "Microsoft Antimalware");
foreach(EventLogEntry entry in log.Entries)
{
if(entry.InstanceId == 1116 && entry.TimeWritten > DateTime.Now.Subtract(new TimeSpan(0, 2, 0)))
{
if(entry.Message.Contains(value: fuEndpointProtection.FileName.ToLower()))
{
Label1.Text = "File has been found to be malware and quarantined!";
return;
}
}
}
Label1.Text = path;
When I upload a normal file
When I upload the Eicar test file
The eventlog entry should look like this, which contains details on the affected process, the fact that it is a virus and also some indication on where to get some more information by providing a threat URL.%%860 4.0.1521.0 {872DA7D0-383A-4A18-A447-DC4C7E71785F} 2012-08-12T09:31:18.362Z 2147519003 Virus:DOS/EICAR_Test_File 5 Severe 42 Virus http://go.microsoft.com/fwlink/?linkid=37020&name=Virus:DOS/EICAR_Test_File&threatid=2147519003 3 2 3 %%818 D:\Windows\System32\inetsrv\w3wp.exe NT AUTHORITY\NETWORK SERVICE containerfile:_F:\sitesroot\0\eicar_com.zip;file:_F:\sitesroot\0\eicar_com.zip->(Zip);file:_F:\sitesroot\0\eicar_com.zip->eicar.com 1 %%845 1 %%813 0 %%822 0 2 %%809 0x00000000 The operation completed successfully. 0 0 No additional actions required NT AUTHORITY\SYSTEM AV: 1.131.1864.0, AS: 1.131.1864.0, NIS: 0.0.0.0 AM: 1.1.8601.0, NIS: 0.0.0.0Okay, so this is very tamed example but it does prove the concept. In the real world you may even want to have a proper gateway which acts as a proxy and then forwards the file onto a "checked" store if it succeeds. We looked at the two ways you can check to see whether the file has been treated as malware. The first, checking to see whether the file has been deleted from it's location is too non-deterministic because although "real time" means real time we don't want to block and wait and timeout on this. The second is better because we will get a report if it's detected. This being the case, a more hardened version of this example will entail building a class which may treat the file write as a task and asynchronously ping back the user if the file has been treated as malware - something like this could be written as an HttpModule or ISASPI filter pursue the test and either continue with the request or end the request and return an HTTP error code to the user with a description of the problems with the file.
Happy trails etc.
Andy Cross
azurecoder
Beth Martin
4 comments
Do you want to comment?
Comments RSS and TrackBack URI
Trackbacks