Microsoft ran the benchmark tests in response to what it believes were IBM's unsubstantiated claims about the benefits of running Linux on its mainframe hardware.
Microsoft general manager of platform strategy Martin Taylor said, "IBM was making claims and customers were trying to figure things out and didn't have the facts in front of them."
The benchmark results contradict IBM's contention that a z900 mainframe processor running Linux could perform as well as three or four Intel processors running Windows at the same clock speed, Taylor claimed
"One Linux image on one zSeries CPU performed at about the same as Windows NT 4 on one 900MHz Intel processor. Windows 2003 outperformed it. No question."
However, Microsoft was unable to reveal at least one important fact relating to today's numbers: the source of IBM's alleged claim that a z900 processor running Linux could perform as well as three or four Intel processors running Windows.
Microsoft maintained that IBM sales staff were making that claim to customers, but was unable to provide any evidence of this. An IBM spokesman contacted about the issue knew nothing about this specific claim.
One industry observer welcomed Microsoft's benchmark numbers. "Microsoft has made a genuine contribution," said Rudy de Haas, an IT consultant who has written about Linux's performance on the mainframe under the pseudonym Paul Murphy.
De Haas has criticised IBM in the past for not providing information on Linux's mainframe performance. "There are several things you could use to raise questions about what Microsoft did, but the real bottom line is that no one else has done it," , he said.
The reason Linux mainframe benchmarks are so scarce is that they simply may not matter to customers, according to IDC analyst Dan Kusnetzky.
"The bigger question is, why would people consider putting Linux on the mainframe? Performance might be a secondary or tertiary issue," he said. "It may be an attempt to lower the cost of maintenance, and the raw performance may have been good enough for the task at hand."
An IBM spokesman said, "We typically don't run industry standard performance benchmarks for any software on the mainframe. The value of the mainframe is based on its ability to securely run multiple applications on a single platform as opposed to purely seeking outstanding performance of one application on a single platform."
Microsoft's benchmarks were conducted by the Veritest division of Lionbridge Technologies under contract to Microsoft. They can be found at http://www.veritest.com/clients/reports/microsoft/ms_performance_updated.pdf
Robert McMillan writes for IDG News Service