You know that technology helps your organisation to
remain competitive, agile and effective - but can you prove it?
Demonstrating the value of technology investment is a constant
challenge for IT directors. However, only a third of senior
executives surveyed by Forrester Research earlier this year said
they had a formal framework in place for measuring the value that
technology delivers to their business.
The problem is that many IT departments concentrate on the initial
cost of technology or on operational performance, says Tony Lock,
chief analyst at Bloor Research. "Very few people really look at
the value returned by technology," he says. "Partly that is because
it is a difficult thing to do, but also people do not really
understand how the business processes are affected by IT."
However, an increasing number of IT departments are being forced to
measure the value of technology investments, according to Craig
Symons, an analyst at Forrester Research. The company's research
suggests 39% of companies without a formal IT measurement framework
are currently developing a system to measure the value of
technology, and another 15% are considering it.
The drive to measure IT's value is down to two factors, Symons
believes. First, as companies increase overall technology spending,
they are looking for greater transparency and, second,
organisations are implementing more stringent governance practices
across the business, including IT.
"We are increasingly seeing boards insist on things like budget
review and approval, project portfolio management and alignment of
business and IT," says Symons.
Having a formal measurement framework means creating a consistent
way of evaluating current and future IT projects in terms of how
they affect the business.
Having a consistent process usually improves an IT department's
overall performance and allows the department to prove its worth to
senior executives, says Lock. "Without it, you are open to IT
becoming not just a cost centre, but also a blame centre," he
says.
However, implementing a measurement framework is not an overnight
job. Analyst firm Gartner advises allowing at least 12 months to
select and roll out an appropriate framework. "It is vital to
choose the right measurement for your organisation," says Gartner
analyst Lars Mieritz. "And that requires an understanding of what
your business perceives to be of value."
Computer Weekly asked the experts how IT directors can choose
between the four most popular IT measurement frameworks in use
today: custom-developed tools, balanced scorecard, return on
investment and Six Sigma. Here is what they said:
Custom-developed tools
About half of all IT departments rely on custom-developed systems
to measure the value of their IT investments, according to
Forrester - and with good reason. "A custom-developed system can be
tailored to reflect business outcomes that are unique to your
organisation," says Symons.
If your organisation does not have the resources to develop a
measurement framework internally, there are many commercial tools
that will do the hard work for you. Euclid and ITM Software, for
example, both sell tools specifically for measuring and reporting
on IT using web-based dashboards and links to existing reporting
packages. Many consultants and analyst firms, including Gartner,
have also developed measurement frameworks that are available to
clients.
Before using a custom-developed system, it is important to
negotiate with the business on which outcomes will deliver value.
Most systems will measure operational metrics, such as availability
and response time, but it is also important to consider whether you
also need to measure less quantifiable factors such as customer
satisfaction or corporate branding. Consider too, how many things
you measure: 20 is sufficient, but 50 different outcomes may
confuse people.
The other important thing to remember is to continue using your
measurement framework once the technology has been deployed. "The
vast majority of companies using informal measurement techniques
forget about them once they have got the money from the board,"
says Mieritz.
"They are too busy planning the next project and the problem is
that nobody monitors whether the promised value was actually
delivered."
Balanced scorecard
Balanced scorecard is one of the most well established formal
measurement frameworks, and is used by 25% of businesses
globally,
according to Forrester.
The technique relies upon a set of metrics that maps operating-unit
performance to corporate financial objectives. Different metrics
can be weighted depending on the priorities of the organisation,
making balanced scorecard reasonably flexible, says Lock. "Overall,
balanced scorecard is probably the easiest to understand
measurement framework, and the most adaptable," he says.
However, a balanced scorecard is only as effective as the baseline
measurements it works from, says Mieritz. "If you cannot benchmark
a particular performance indicator accurately on day one, any
subsequent measurements will be useless," he says.
To help companies improve the reliability of balanced scorecards,
suppliers are developing software tools that link into enterprise
applications and automatically generate scorecard data. For
example, Panasonic uses data produced by a portfolio analysis
system from ProSight to create balanced scorecards, based on 38 key
objectives, such as maximising business unit productivity and
better understanding customers.
Return on investment
Perhaps the most widely used measurement technique in IT
departments is return on investment (ROI). However, the experts are
universal in their criticism of its ability to truly demonstrate
the value of technology investment.
"Three-quarters of CIOs admit they do not achieve the results
promised by ROI analysis," says Mieritz. "Why should the business
put any faith in the IT department when those are the results they
are getting?"
The problem is that ROI is almost impossible to calculate
accurately, says Lock, particularly when companies rely on free ROI
calculators or basic tools. "ROI is really something they offer you
free with Excel," says Lock. "It is really only suitable for the
most basic, straightforward projects."
To get the best from ROI calculation, Lock suggests companies
should incorporate business outcomes along with operational
performance improvements, and collaborate with the business to put
a value on such outcomes. In addition, look for other companies
that have already created ROI tools for similar projects. "There
are loads of examples of best practice on the web, which save you
reinventing the wheel every time," he says.
However, some projects are unsuitable for ROI analysis. In
particular, Lock advises against relying on ROI for projects where
the benefits come mainly from changed processes or human capital.
"If you use ROI to measure people costs, you are wasting your
time," he says. "Most people costs in a business are in big
buckets, and trying to identify how much time one person spends on
a task is almost always more trouble than it is worth."
Six Sigma
Only one in 10 enterprises use Six Sigma, perhaps because the
measurement framework is so complex. Six Sigma uses data and
statistical analysis to improve business processes and products by
eliminating "defects" which can be anything from a freak flood to a
weak link in the supply chain.
Companies such as GE Finance use Six Sigma to measure the value
delivered by all new IT investments. To pass muster, a process may
need to achieve Six Sigma Level 5, for example, which means it
cannot yield more than 230 defects for every million
repetitions.
Six Sigma can be ideally suited when companies are heavily focused
on customer service and satisfaction. The framework's aim is to
improve both these outcomes. However, achieving it within an IT
environment can be extremely challenging, says Mieritz.
"To achieve Six Sigma, you have to be able to precisely define the
process, measure the process, analyse the effectiveness and then
implement improvements," he says. "That requires a great deal of
communication between the IT and business units, which can be
difficult for some organisations if they are not used to that level
of collaboration."
Case study: homing in on business benefits with ROI
approach
Home interior and fashion group Internacionale relies
exclusively on return on investment (ROI) to measure the value of
new IT investments.
Every new project is measured according to a predefined set of
costs and benefits, which are based on the company's one-, three-
and five-year strategic plans, explains IT controller Isabel
McCabe. "We will then combine those costs and benefits with more
specific factors for each product," she says.
Although the company sometimes uses suppliers' ROI calculators,
most ROI analysis is done internally. It helps that the company's
senior management is relatively familiar with technology, and able
to understand operational metrics in most cases. "The aim is to
come up with something comprehensive, which will be accessible to
the board members who need to approve the investment," says
McCabe.
Building an ROI case begins before any investment is made and
continues through deployment and post-implementation analysis. "We
also have monthly update meetings where we look at what progress
has been made, areas where we might have slipped and whether we are
achieving the figures spelled out in the ROI analysis," says
McCabe.
For more complex projects, Internacionale may supplement an ROI
analysis with another measurement technique, such as service level
agreements, to monitor the cost of a service and the quality
delivered to end-users. The IT department also measures the amount
of support required per business unit, as new technology is
deployed. "The idea is that if we spent 1.5 days annually
supporting a process but now spend half a day, that is of
measurable value to the business," says McCabe.
Case study: benefits calculator presses right buttons at
Dover
The CRM National Programme is one of 22 e-government national
projects set up to help local authorities deliver better services
to
citizens and improve efficiency. One result is a custom "business
benefits calculator" tool, developed in partnership with IT
services company Touchstone and the London Borough of Newham.
The CRM calculator helps councils identify a comprehensive list of
cashable and non-cashable benefits to help them scope the impact of
a customer relationship management project. Dover District Council
has used the tool to measure the value of CRM in its customer care
improvement programme, explains programme manager Graeme
Webb.
"We felt it would be really helpful to have some sort of benchmark
before we started on this project - where would we have to start
and
where would we end up?"
Following a meeting with the council's chief executive, Webb and
his team decided to concentrate on the area of waste management as
applicable to CRM. "It is an area with high volume but low
complexity transaction-wise, and we soon marked out four clear
processes that we thought we could improve."
The council then held a series of workshops, based on the CRM
calculator approach. "Clearly CRM in itself is not going to improve
the citizen's experience; that's down to people, processes and
technologies," says Webb.
"But this approach meant we could plan better for where we needed
to make investment, the new links between front and back office,
and where cultural change might occur. We know which CRM system to
buy now and for what reasons," he says.