Hewlett-Packard and Sun Microsystems have recently upped the ante
in the benchmarking stakes with impressive results for the retested
Superdome server and the latest Ultrasparc chip respectively,
writes Eric Doyle.
Superdome, HP's thoroughbred 9000 series server, has been retested
to the TPC-C (Transaction Processing Council) benchmarking standard
to reveal a transaction processing figure double its previous
result.
The first results showed little difference between Superdome and
comparable systems from IBM and depressed the potential sales of
the system that had been predicted pre-launch.
The company is now trying to prove, in court, that the original
results were deliberately sabotaged by its former employee
Hock-Beng Lim. The charges against him imply that he was paid by an
unnamed third party to ensure a less-than-impressive result.
The latest results are nearer to the degree of performance that HP
expected, but failed to achieve, when the system was launched at
the beginning of this year.
The TPC-C tests are well respected in the industry as an index to
the performance of a hardware/database combination and are devised
to emulate a real-world online transaction processing
environment.
The goal is to reveal comparable results regardless of the
processor and operating system in use. The resultant report reveals
an index figure in transactions per minute (tpmC) from which a
price/ performance indicator, in terms of dollars per transaction,
can be derived.
Critics of the TPC methodology focus on the fact that the test does
not truly represent hardware performance because there is a free
choice of the database used.
Whatever the view of the comparability of individual system
results, HP's latest figures are, without doubt, impressive. The
system differs from previous tests because it uses the
recently-released PA-8700 Superdome with the Oracle 9i database in
a configuration that will not be available until May next year.
In the last test on the $400,000 (£285,000) system, taken in May
this year, the benchmark was 197,024 tpmC, which represents
$43.25/tpmC but this has been increased to 389,434tpmC -
$21.24/tpmC.
This is a vastly superior result compared to IBM's best result so
far, taken in April using an IBM p680 Unix server with the older
Oracle 8 database, where the figures were 220,807tpmC and $34.18/
tpmC.
However, HP is still trailing behind Fujitsu's Sun Solaris- powered
Primepower 2000's August result using a Symfoware Server Enterprise
Edition database, which together gave a tpmC rating of 455,818 but
a less favourable £28.58/tpmC result.
Sun, which currently has only one test result under TPC - Sun
Enterprise 4500 at 67,102tpmC, $25.85/tpmC - prefers to promote its
Spec (Standard Performance Evaluation Corporation) results for
compute-intensive CPU performance.
Using the CPU2000 test suite, Sun's latest Sparc processor
performed surprisingly better than previous iterations.
The Spec tests result in scores, known as a mark, being given under
various categories of test. This allows a comparison to be made
between processors according to which scores higher and the main
categories are compute power when handling integer (whole) numbers
and when using floating point (decimal) numbers.
The Ultrasparc III Cu 1050 processor revealed a results of 610 for
Spec integer and a Spec floating point of 827. This shows a notable
improvement over earlier iterations of the chip of 32% and 72%
respectively.