A study from Forrester Research has concluded that the
Linux operating system is not necessarily more secure than Windows,
with Linux distributors taking longer than Microsoft to patch
security holes, although Microsoft flaws tended to be more
severe.
However, Linux supplier Red Hat said that while Forrester's
underlying figures were sound, its conclusions gave an inaccurate
idea of relative security as they failed to distinguish between
patch times for critical updates and routine, obscure problems.
The report arrives in the midst of a fierce debate around the
relative merits of Linux and Windows, and follows a number of
reports perceived to have been slanted in Microsoft's favour.
Last October, Forrester forbade its customers to publicise
studies they had commissioned. It made the move partly because of
criticism of a report from Forrester subsidiary Giga Research that
found some companies saved money by developing with Windows rather
than Linux. Forrester said it stood by the integrity of the study,
but had erred in allowing Microsoft to use it in anti-Linux
advertising.
Forrester's report may lend credibility to Microsoft's efforts
to play down security concerns about its software. A new tactic in
that battle has been to compare how long it takes for various
operating system vendors to patch flaws - the "days of risk" for
each operating system.
Microsoft's argument is simple, said Bradley Tipp, Microsoft’s
National Systems Engineer for the UK. "Open-source systems are
likely to be at risk for more days than Windows systems," he
claimed last autumn.
Forrester found that, between 1 June 2002 and 31 May 2003,
Microsoft had the lowest average "all days of risk", the time
between the public disclosure of a patch and the time that patch is
released by the operating system maintainer, compared with the Red
Hat, Debian, MandrakeSoft and SuSE Linux AG distributions.
Microsoft took on average 25 days to release a patch; Red Hat
and Debian 57, SUSE 74 and MandrakeSoft 82, Forrester said.
"Microsoft’s average of 25 days between disclosure and release of a
fix was the lowest of all the platform maintainers we evaluated,"
wrote analyst Laura Koetzle in the report. "Microsoft also
addressed all of the 128 publicly disclosed security flaws in
Windows during our 12-month evaluation period."
Koetzle noted, however, that 67% of Windows flaws had been rated
"critical" under the US National Institutes for Standards and
Technology's ICAT project standard for high-severity
vulnerabilities, compared with 63% for SuSE, 60% for MandrakeSoft,
57% for Debian and 56% for Red Hat.
Since Linux distributions are compilations of large numbers of
independent components, the study also examined lag-times between
the release of a patch for a Linux component and the release of the
same fix by the operating system supplier, what Forrester called
"distribution days of risk".
Debian scored best in this metric, with 32 days, followed by Red
Hat with 47 days, SuSE with 54 days and MandrakeSoft with 56
days.
Red Hat said the figures Forrester relied on for Linux
distributions were above reproach, as various Linux distributors
worked with the analyst firm on weeding out errors. However, it
claimed the conclusions drawn from those figures are nearly
useless.
"A simple average doesn't give you a good picture at all," said
Red Hat security response team lead Mark Cox. "It wastes the work
put into the raw data."
The figures Forrester uses for "all days of risk" are arrived at
by averaging the number of days needed to fix a flaw, without
distinguishing between critical flaws and harmless ones. So if a
supplier took six months to patch a low-risk bug, it would make it
appear to have a slow security response time overall, even if all
critical bugs had been fixed instantly.
Using Microsoft's own definition of a critical flaw as a bug
which could allow a worm to propagate without user interaction,
only 13 Red Hat vulnerabilities were critical during the one-year
time period, and they took an average of just over a day to fix,
Cox said.
"If you add denial of service attacks and privilege escalations,
there were 47 issues in total, which took seven days on average to
fix," he added.
"We fix issues that are critical to users first," he said. "When
a remote exploit comes out, we drop everything to make sure it
comes out quickly. That's more important than a bug in some obscure
package no one uses. The report really doesn't take that into
account. It's a shame because the raw data is there."
Cox also took issue with the perception that there is
necessarily a lag between a module patch and a distribution patch -
Forrester's "distribution days of risk". If a bug is critical, it
will be released by the Linux supplier immediately, he said; if
module maintainers have not yet released a patch, Red Hat and other
distributors do it themselves.
Cox said Red Hat is taking measures to deal with the lag time
between the release of a patch and users' implementation of it,
including making each Red Hat machine slightly different and a
kernel program called exec-shield. Red Hat and other distributors
are also participating in the Security Enhanced Linux project.
Matthew Broersma writes for
Techworld.com