A few short days ago I asked the question “Do the anti-malware controls built into Windows Vista mean that we can begin to think about reducing the amount we spend on third party desktop AV products?” A new report comparing a number of AV products seems to have answered that question for us. Read it here: http://www.av-comparatives.org/seiten/ergebnisse/report13.pdf. The report places Microsoft’s OneCare desktop anti-virus product in last place for each of the tests performed, and on the surface appears to be a negative testimony to Microsoft’s ability within this area. There’s been plenty of commentary already reported such as this article here in Computer World.
Personally I think we need to review this report with some caution. There is no detail as to what configurations or settings were used to perform the evaluation and neither do we get much insight into the test methodology used. On the other hand, OneCare has already passed the ICSA labs certification tests – you can find the evidence of this in the July report link from here.
Conversely, at around the same time as ICSA were performing their lab tests, another report was released by Agnitum. This one is pretty damning, stating as it does that “OneCare firewall failed all but the simplest leak tests and does not offer even the most basic intrusion detection capability, leaving users’ PCs wide open to being hijacked into a botnet.”
So, what’s the real story? I generally put a fair amount of faith in the ICSA lab tests – they are an accepted, reputable standard while other reports, on the surface, appear subjective. For instance, Agnitum discuss weaknesses in the product but then go on to talk about industry reactions to Microsoft’s entry into the desktop security marketplace. Their report is also high level, making no mention of how it was installed, the configurations used, or the system it was installed onto.
Fact of the matter is that Microsoft have stuck their wealthy necks out into a market place dominated by aggressively pitched products and are likely to take a fair proportion of the market share aware from the established vendors in this field. That’s not a bad thing – competition is good for the consumer and we might start to see some innovation within, what I think, has become a rather staid and predictable field of security.