Linux lags Windows in new security report

A report released today indicates Windows Server 2003 may actually be more secure than its most popular Linux competitor when it comes to vulnerabilities and the time it

The Net is just rife with opinion on security of Windows and Linux but there's very little key decision data points out there. Herbert H. Thompson, research director, Security Innovation ,

But well before the paper's official release, members of the IT security community have questioned the comparison, with some slamming the researchers' methodology and others the Microsoft connection -- the software giant funded the research behind the favorable findings.

"The fact that Security Innovations [which produced the paper] retained 'editorial control' doesn't help; if Microsoft is paying the bills, there can be all sorts of nonverbal pressure behind the scenes. It isn't like it was 'co-funded' by both Microsoft and Red Hat," said Michael D. "Mick" Bauer, senior editor of Linux Journal and director of value-subtracted services for Wiremonkeys.org.

He also questioned the narrow focus. "This study appears to be more concerned with vulnerability counts and patch-release cycles than in actual security or securability. Certainly, if Microsoft has reduced the amounts of bugs in [its] software and gotten faster at patching bugs, that's great. But the bug-patch rat race is only one part of a much more complicated security picture, and the way I see it, Linux still has compelling advantages from a security standpoint."

Comparable metrics used, authors say

Such a reaction was anticipated by authors Richard Ford, Herbert H. Thompson and Fabien Casteran. They intentionally ignored threat profiles in favor of inherent vulnerabilities in Windows Server 2003 and two versions of Red Hat Enterprise Linux 3.0. The goal, they said, is to provide a security metric for IT professionals to apply to their own software shopping.

"I don't think people should make adoption decisions purely based on the results, but I think it does at the very least give decision makers and diehards on either side, or even the neutral people, a chance to look beyond hype and speculation and look at hard numbers," said Thompson, director of research at Melbourne, Fla.-based Security Innovation Inc., the application security provider that produced the report.

Thompson denies Microsoft's money influenced results but admits that's a source of contention for a lot of people. "We've gotten funding from Microsoft and as a result of that people have come back and said this automatically must not be relevant and fair and balanced. That's one reason our mission has been to be completely transparent in the methodology."

Microsoft has funded similar security studies, based on customer requests. This also is not the first time a Microsoft-favorable report's come under attack. Last year' an independently produced Forrester Research study tconcluded Windows had a lower average total days of risk than the four most popular Linux distributions. Another, also by Forrester, had shown Windows had a lower total cost of ownership. Both reports came under similar attack.

In the Security Innovation report, the trio took requirements for three typical enterprise Web server environments and scrutinized known vulnerabilities and subsequent patches. The Windows Server 2003 platform included ASP.NET for scripting, a SQL Server 2000 database server and Microsoft Internet Information Services 6.0 Web server. Any function was accepted by default during installation (assuming many admins just keep clicking the Next button during the process). On the Linux side, the team used two different configurations for Red Hat Enterprise Linux 3.0. Both ran PHP for scripting, a MySQL database server and an Apache Web server. But one version included high modularity, where essentially the researchers installed whatever Red Hat had available; the other was minimally configured to include only core components.

Key findings of the research

Among the findings: During calendar year 2004, the Windows platform recorded 52 vulnerabilities, while the default Linux installation included 174 vulnerabilities and the bare-boned version had 132 known flaws. Because of disparate severity ratings among vendors, the researchers used the more neutral ICAT system from the National Institute for Standards and Technology to rank a flaw's criticality. Using that government-funded system, the Windows configuration had 33 serious holes, compared to 48 for the minimally configured Linux machines and 77 on the loaded Linux box.

The other metric measured how much time lapsed between public disclosure, such as through announcements on Bugtraq, and a patch release. Researchers referred to the gap as "days of risk." In Windows, the average was 31.3 days; in Linux it was 69.6 days for minimally configured Red Hat and 71.4 for the default installation.

In addition, all three configurations contained holes left exposed for more than 90 days from disclosure to fix release. Seven were found in Windows Server 2003, with five designated as "highly severe" by ICAT. Four of those holes were in the Internet Explorer Web browser. In the minimally configured Red Hat, 31 holes were found, seven of them highly severe and five others not rated by ICAT at the time of the study. Eleven vulnerabilities were in the operating system kernel; followed by MySQL with five.

Thompson and Ford gave a preview of their report at February's RSA Conference, in which numerous audience members challenged their choices and conclusions. At the presentation, Ford defended their methodology. "We think it's thought through. We think it's pretty balanced," the Linux enthusiast had said.

A lack of hard data available

On Monday, Thompson said suggestions and comments since RSA were incorporated into the final draft. The research, he stressed, is intended to aid IT managers weighing software purchases as well as shed light on what vendors and user communities are doing to reduce the number of security flaws in these products. "There's so much speculation out there," he said. "The Net is just rife with opinion on security of Windows and Linux but there's very little key decision data points out there, and that's one of the things we hope to provide."

But people like Bauer say the results remain unfair comparisons.

"Most of us in the Linux security community have been saying for years that the average Linux distribution -- Red Hat, SuSE, etc. -- isn't terribly secure 'by default.' Good security comes from careful configuration, not by running an installer," he said.

Jay Beale, lead developer of the Bastille Linux Project, questioned the choice of vulnerabilities. "They're focusing on high severity vulnerabilities. A local privilege escalation exploit is high severity, which is true. But they argue that high severity vulnerabilities should be fixed fast. Actually, while local priv escalation vulns are high severity, they're not high risk. And so neither vendor fixes them very quickly."

Bauer did give Microsoft a nod for recent improvements in its software security, including more timely patch releases. "But I still like Linux better from a security standpoint," he said. "Even though this is less true every year, I still find many of the choices that Microsoft makes for me to be maddening, such as the way Windows handles digital certificates. With Linux I simply have more choice in determining how my system behaves, and to be security-conscious is to be a control freak."

News editor Shawna McAlearney contributed to this report.