You know that technology helps your organisation to remain competitive, agile and
effective - but can you prove it? Demonstrating the value of technology investment is a constant
challenge for IT directors. However, only a third of senior executives surveyed by Forrester
Research earlier this year said they had a formal framework in place for measuring the value that
technology delivers to their business.
The problem is that many IT departments concentrate on the initial cost of technology or on operational performance, says Tony Lock, chief analyst at Bloor Research. "Very few people really look at the value returned by technology," he says. "Partly that is because it is a difficult thing to do, but also people do not really understand how the business processes are affected by IT."
However, an increasing number of IT departments are being forced to measure the value of technology investments, according to Craig Symons, an analyst at Forrester Research. The company's research suggests 39% of companies without a formal IT measurement framework are currently developing a system to measure the value of technology, and another 15% are considering it.
The drive to measure IT's value is down to two factors, Symons believes. First, as companies increase overall technology spending, they are looking for greater transparency and, second, organisations are implementing more stringent governance practices across the business, including IT.
"We are increasingly seeing boards insist on things like budget review and approval, project portfolio management and alignment of business and IT," says Symons.
Having a formal measurement framework means creating a consistent way of evaluating current and future IT projects in terms of how they affect the business.
Having a consistent process usually improves an IT department's overall performance and allows the department to prove its worth to senior executives, says Lock. "Without it, you are open to IT becoming not just a cost centre, but also a blame centre," he says.
However, implementing a measurement framework is not an overnight job. Analyst firm Gartner advises allowing at least 12 months to select and roll out an appropriate framework. "It is vital to choose the right measurement for your organisation," says Gartner analyst Lars Mieritz. "And that requires an understanding of what your business perceives to be of value."
Computer Weekly asked the experts how IT directors can choose between the four most popular IT measurement frameworks in use today: custom-developed tools, balanced scorecard, return on investment and Six Sigma. Here is what they said:
About half of all IT departments rely on custom-developed systems to measure the value of their IT investments, according to Forrester - and with good reason. "A custom-developed system can be tailored to reflect business outcomes that are unique to your organisation," says Symons.
If your organisation does not have the resources to develop a measurement framework internally, there are many commercial tools that will do the hard work for you. Euclid and ITM Software, for example, both sell tools specifically for measuring and reporting on IT using web-based dashboards and links to existing reporting packages. Many consultants and analyst firms, including Gartner, have also developed measurement frameworks that are available to clients.
Before using a custom-developed system, it is important to negotiate with the business on which outcomes will deliver value. Most systems will measure operational metrics, such as availability and response time, but it is also important to consider whether you also need to measure less quantifiable factors such as customer satisfaction or corporate branding. Consider too, how many things you measure: 20 is sufficient, but 50 different outcomes may confuse people.
The other important thing to remember is to continue using your measurement framework once the technology has been deployed. "The vast majority of companies using informal measurement techniques forget about them once they have got the money from the board," says Mieritz.
"They are too busy planning the next project and the problem is that nobody monitors whether the promised value was actually delivered."
Balanced scorecard is one of the most well established formal measurement frameworks, and is used by 25% of businesses globally,
according to Forrester.
The technique relies upon a set of metrics that maps operating-unit performance to corporate financial objectives. Different metrics can be weighted depending on the priorities of the organisation, making balanced scorecard reasonably flexible, says Lock. "Overall, balanced scorecard is probably the easiest to understand measurement framework, and the most adaptable," he says.
However, a balanced scorecard is only as effective as the baseline measurements it works from, says Mieritz. "If you cannot benchmark a particular performance indicator accurately on day one, any subsequent measurements will be useless," he says.
To help companies improve the reliability of balanced scorecards, suppliers are developing software tools that link into enterprise applications and automatically generate scorecard data. For example, Panasonic uses data produced by a portfolio analysis system from ProSight to create balanced scorecards, based on 38 key objectives, such as maximising business unit productivity and better understanding customers.
Return on investment
Perhaps the most widely used measurement technique in IT departments is return on investment (ROI). However, the experts are universal in their criticism of its ability to truly demonstrate the value of technology investment.
"Three-quarters of CIOs admit they do not achieve the results promised by ROI analysis," says Mieritz. "Why should the business put any faith in the IT department when those are the results they are getting?"
The problem is that ROI is almost impossible to calculate accurately, says Lock, particularly when companies rely on free ROI calculators or basic tools. "ROI is really something they offer you free with Excel," says Lock. "It is really only suitable for the most basic, straightforward projects."
To get the best from ROI calculation, Lock suggests companies should incorporate business outcomes along with operational performance improvements, and collaborate with the business to put a value on such outcomes. In addition, look for other companies that have already created ROI tools for similar projects. "There are loads of examples of best practice on the web, which save you reinventing the wheel every time," he says.
However, some projects are unsuitable for ROI analysis. In particular, Lock advises against relying on ROI for projects where the benefits come mainly from changed processes or human capital. "If you use ROI to measure people costs, you are wasting your time," he says. "Most people costs in a business are in big buckets, and trying to identify how much time one person spends on a task is almost always more trouble than it is worth."
Only one in 10 enterprises use Six Sigma, perhaps because the measurement framework is so complex. Six Sigma uses data and statistical analysis to improve business processes and products by eliminating "defects" which can be anything from a freak flood to a weak link in the supply chain.
Companies such as GE Finance use Six Sigma to measure the value delivered by all new IT investments. To pass muster, a process may need to achieve Six Sigma Level 5, for example, which means it cannot yield more than 230 defects for every million repetitions.
Six Sigma can be ideally suited when companies are heavily focused on customer service and satisfaction. The framework's aim is to improve both these outcomes. However, achieving it within an IT environment can be extremely challenging, says Mieritz.
"To achieve Six Sigma, you have to be able to precisely define the process, measure the process, analyse the effectiveness and then implement improvements," he says. "That requires a great deal of communication between the IT and business units, which can be difficult for some organisations if they are not used to that level of collaboration."
Case study: homing in on business benefits with ROI approach
Home interior and fashion group Internacionale relies exclusively on return on investment (ROI)
to measure the value of new IT investments.
Every new project is measured according to a predefined set of costs and benefits, which are based on the company's one-, three- and five-year strategic plans, explains IT controller Isabel McCabe. "We will then combine those costs and benefits with more specific factors for each product," she says.
Although the company sometimes uses suppliers' ROI calculators, most ROI analysis is done internally. It helps that the company's senior management is relatively familiar with technology, and able to understand operational metrics in most cases. "The aim is to come up with something comprehensive, which will be accessible to the board members who need to approve the investment," says McCabe.
Building an ROI case begins before any investment is made and continues through deployment and post-implementation analysis. "We also have monthly update meetings where we look at what progress has been made, areas where we might have slipped and whether we are achieving the figures spelled out in the ROI analysis," says McCabe.
For more complex projects, Internacionale may supplement an ROI analysis with another measurement technique, such as service level agreements, to monitor the cost of a service and the quality delivered to end-users. The IT department also measures the amount of support required per business unit, as new technology is deployed. "The idea is that if we spent 1.5 days annually supporting a process but now spend half a day, that is of measurable value to the business," says McCabe.
Case study: benefits calculator presses right buttons at Dover
The CRM National Programme is one of 22 e-government national projects set up to help local
authorities deliver better services to
citizens and improve efficiency. One result is a custom "business benefits calculator" tool, developed in partnership with IT services company Touchstone and the London Borough of Newham.
The CRM calculator helps councils identify a comprehensive list of cashable and non-cashable benefits to help them scope the impact of a customer relationship management project. Dover District Council has used the tool to measure the value of CRM in its customer care improvement programme, explains programme manager Graeme Webb.
"We felt it would be really helpful to have some sort of benchmark before we started on this project - where would we have to start and
where would we end up?"
Following a meeting with the council's chief executive, Webb and his team decided to concentrate on the area of waste management as applicable to CRM. "It is an area with high volume but low complexity transaction-wise, and we soon marked out four clear processes that we thought we could improve."
The council then held a series of workshops, based on the CRM calculator approach. "Clearly CRM in itself is not going to improve the citizen's experience; that's down to people, processes and technologies," says Webb.
"But this approach meant we could plan better for where we needed to make investment, the new links between front and back office, and where cultural change might occur. We know which CRM system to buy now and for what reasons," he says.
This was first published in August 2005