SYSmark is distributed by a consortium called BAPCo, or the Business Application Performance Corporation. Members of the group include Intel, Dell Computer, IBM, Hewlett-Packard, Microsoft and AMD, which recently joined the consortium.
PC buyers rely on benchmarking software to determine which processor will suit their overall computing needs. However, charges of bias and manipulation over the past few days have brought to light the discrepancies in benchmarking software and in its effectiveness to users.
AMD's processors outperformed Intel's on the 2001 version of the SYSmark benchmark. However, in this year's tests, AMD claimed that certain application tests had been removed and replaced with tests that favour Intel's Pentium 4 processors.
The company contended that the Intel's presence as the only major microprocessor vendor in BAPCo has caused the benchmarking software to drift toward Intel's philosophy of real-world performance without a balancing viewpoint from AMD, Speed said.
Specifically, AMD charged that of 13 filters measuring the performance of Adobe Photoshop software in the 2001 version of the benchmark, eight that favoured the Athlon processor were replaced with new filters that favoured the Pentium 4 in the 2002 version. The 2002 benchmark also uses multiple instances of three filters that favoured Intel and adds three new filters favouring the Pentium 4, Speed said.
The portion of the benchmark that measures Microsoft Excel performance is heavily tilted toward sorting, a procedure that does not represent the most common uses of Excel, Speed said. The Pentium 4 sorts data in Microsoft Excel faster than AMD's Athlon chips, according to the 2001 benchmarking results.
AMD also claimed that tasks favouring AMD for Microsoft's Access database were almost completely removed. The Access results from the 2001 benchmark favoured Athlon processors, but the 2002 version draws much less of the overall score from Access results.
Intel would not comment on the specific allegations.