Most companies are getting poor value from the tens of
millions of pounds spent on benchmarking the performance of their
IT, according to a new survey.
The survey, by research company OTR Group, questioned 300 users,
outsourcing suppliers and benchmarking companies across Europe and
found that benchmarking internal IT services was not an effective
way to measure the value of IT investment.
Respondents also said that benchmarking internal IT did not give
them practical advice on how to improve their service.
Benchmarking has become a standard way for UK organisations to
measure the effectiveness of their IT department or outsourcing
suppliers.
Benchmarking judges a particular service based on criteria
including its costs, daily performance and a comparison with other
IT departments or outsourcing suppliers.
"The survey found widespread dissatisfaction with benchmarking,"
said Andrew Pooley, group commercial director at research company
OTR.
"Benchmarking can provide value for money if the right things are
measured, but many users need to review what they are doing."
The survey found that benchmarking worked well when used on certain
types of deal. Benchmarking of "open book" outsourcing agreements,
in which users have access to their suppliers' accounts, was judged
particularly effective.
Organisations benchmarking these types of outsourcing agreements
saved an average of 30% on IT costs through negotiating more
favourable terms and conditions with suppliers.
'Lack of insight'
Scottish & Newcastle used a benchmarking consultancy to measure
its IT costs and look for best practices before beginning a large
project. The brewing and leisure group had previously used its own
staff to benchmark IT services.
Scottish & Newcastle said benchmarking helped identify which
supplier would give the best service at the lowest cost.
But it was less happy with other parts of the benchmarking service.
It said that the bench-marking service did not "deliver any
insight" into whether their IT plans were along the right lines, or
help it identify best practices to apply to its IT department.
Councils prove value for money
In the public sector benchmarking has become a useful tool for
councils which have to show inspectors their services are providing
value for money. According to Socitm, the association of local
authority IT managers, more than 100 UK local authorities currently
use its benchmarking service.
The benchmarking service measures in-house and outsourced
services on performance indicators, such as cost and reliability.
Council performance is compared to similar organisations using
information held on a Socitm database.
Martin Greenwood, Socitm Insight manager, said, "Benchmarking
can work well when used to judge the effectiveness of an IT
department. The important thing is to use the right measures, for
example, user satisfaction."