Four major Linux distributors have attacked a Forrester Research report which claimed that Microsoft outperformed them in responding to and fixing security flaws.
Forrester's report, titled "Is Linux More Secure than Windows?" looked at how Microsoft and the four Linux suppliers responded to reports of security flaws over a 12-month period. The document gave Microsoft the highest marks for its "responsiveness" and its "thoroughness" in dealing with reported security vulnerabilities.
Linux distributors Debian, MandrakeSoft, Red Hat and SuSE Linux claimed that the report had "extremely limited real-world value" for users.
"It's bogus in its current form," said Joey Schulze, a member of Debian's security team.
While the data that the analysis is based on is accurate, the conclusions are not, said Vincent Danen, security update manager at MandrakeSoft. By treating supplier responses to all vulnerabilities as equal, the Forrester report failed to measure the much better record of Linux distributors when it comes to particularly serious flaws, he added.
Linux suppliers typically treat flaws on a case-by-case basis, with high-risk flaws getting a higher priority than the low-risk ones, Danen said. The response to a flaw is based on risk assessments made by each distributor and may not always coincide with the assessment made by a third party such as National Institute of Standards and Technology (Nist), he said.
"Our users will know that for critical flaws, we can respond within hours," SuSE Linux said in a statement.
By focusing purely on quantitative analyses, the Forrester report fails "to differentiate between both the seriousness of the flaws and, more importantly, the quality of the fixes," SuSE added.
In September, Forrester drew flak from users over a report that showed Microsoft development platforms having a substantial cost advantage over Linux/J2EE for portal applications, since that report was actually funded by Microsoft.
Laura Koetzle, author of the latest Forrester report, defended her company's analysis of the data, insisting that all suppliers studied in the report were measured equally using publicly available and widely accepted vulnerability rating measures from Nist.
"Microsoft did not pay for this report," she added.
Jaikumar Vijayan writes for Computerworld