Analyst firm Giga Information Group has advised users to conduct
their own Java application server performance tests as suppliers
have been unwilling to provide comparable test data.
The ECPerf benchmark, released last year by Sun, was supported by a
number of Java application server suppliers including BEA Systems,
IBM, Sun Microsystems, Oracle, Sybase, iPlanet, Hewlett-Packard,
IONA, Informix Software, Borland and ATG.
But according to Giga, Java application server suppliers have been
unwilling to publish an "apples to apples" (comparable hardware and
software configuration) ECPerf benchmark.
Giga advised any company with both the required expertise/resources
and funding to perform the ECPerf benchmark on their own corporate
hardware, database and operating system specific configurations in
order to get the most accurate benchmark comparisons.
It noted that to perform such benchmarks users would need at least
one to two J2EE technologists, one system/application server
administrator and one database administer (DBA) for a minimum of
two to three weeks.
Giga analyst John Meyer said: "Once the initial set of internal
benchmarks have been completed, companies should invite [in] the
application server vendors to allow them to attempt to tweak the
performance of the configurations, assuring the most accurate
benchmark possible."
Meyer said that in most cases the tier-one application server
vendors would be much more likely to provide resources to tweak an
existing ECPerf benchmark for their application server compared to
running a complete ECPerf benchmark for a prospective
customer.
The entire ECPerf benchmark can be downloaded from Sun at
java.sun.com/j2ee/ecperf/download.html.