Government measures

The government sees IT as pivotal to its plans to improve public services, but the Treasury is in desperate need of benchmarks to...

The government sees IT as pivotal to its plans to improve public services, but the Treasury is in desperate need of benchmarks to judge its performance. Electronic service delivery and the balanced scorecard could be a way forward, reports Phil Cain

In the private sector, any failure of IT to keep its project promises can lead to calls from disgruntled shareholders and harassed board members for it to demonstrate its worth. And in the public sphere, the government is lending its considerable weight to this pressure. Between the 2000 and 2002 spending reviews it approved £6bn spending on what it calls "modern technology". And in July's spending review this year, in which the Treasury set out departmental spending priorities for the next three years, it reiterated its belief in IT's pivotal role in public service improvement.

But it also placed a new focus on getting a return on its investment, saying that so far "little emphasis [in government] has been given to the efficiencies that could be delivered by realising the full benefit of these investments".

Meanwhile, the Gershon Efficiency Review, a simultaneously released study which informed the spending review, suggests these savings could come from migrating customers to new electronic channels and restructuring internal processes to reduce administrative costs.

Local government, which accounts for about half the IT spending and is responsible for more than three-quarters of government services, is where the Treasury most eagerly wants to see its money back. The review set a target of achieving, "efficiency savings in local government of 2.5% a year to deliver £6.45bn of efficiencies and productivity improvements by 2007-2008, releasing additional resources to front-line services".

But how can this be achieved and who will know if it has been? Although the Treasury has a keen eye on the costs, the benefits and savings are much harder to pin down. Without knowing what these are, who is to say if the target has been achieved or, on a smaller scale, if a particular project should be funded or axed?

Solving this riddle will mean local government IT project and programme managers will all face their share of Treasury heat. "The agenda has changed to improving efficiency and the process for measuring your efficiency," says Chris Taylor at the Improvement and Development Agency, a government-funded local government advisory service.

To complicate matters, the Audit Commission, the agency in charge of measuring local government performance in England and Wales, scrapped assessments of IT departments in 2002. Like everyone else in local government, it is left wondering how to measure the performance of public sector organisations. How will IT's contribution be recognised? "The Audit Commission has not yet sorted out how that process will work," Taylor says.

From the point of view of public sector IT management, "there is probably the potential to get these levels of efficiency," says John Serle, chair of the best value group at local government IT managers' association Socitm.

But he also believes the review fails to acknowledge the political realities of local government. Although private sector service providers such as banks can easily derive cost savings from new, cheaper electronic channels by shutting outlets, it is more difficult for councils. "We are talking about schools, libraries and police stations," Serle says. Making technology-enabled changes does not depend so much on technology, but "having the political will to carry it through".

Glyn Evans, chair of Socitm's information age government group, says that part of the problem is that the review's logic is more relevant to the few high-volume services in central government, such as vehicle licensing. "A lot of local government services are not like that. Social care, for example, is targeted at what an individual needs," he says. Rather than attempt to derive a small percentage saving per transaction, "the way to get efficiencies in local government is to do things radically differently".

But the Office of the Deputy Prime Minister (ODPM) has, according to Taylor, provided the beginnings of an answer to measuring progress in its requirement that councils report progress on e-government projects through annual "implementing electronic government" statements. In so doing it has made the crucial step that links funding to a funding mechanism. Beginning four years ago as an annual requirement, the ODPM's door is now open to reporting through an online site called Electronic Service Delivery Tooling.

The potential for a switch to near continuous, rather than annual, measurement may fit in with current trends in project management. "You do not just write a big spec and say, 'There it is!' It is a more interactive process." Project managers these days take shorter and quicker steps, says Taylor.

Although there might be a mechanism for measurement, what should it measure? Most traditional techniques tend to look on IT investment in terms of financial return on an asset, based on the theory of capital markets, which does not fit with the public sector ethos.

"I am interested in the balanced scorecard approach," Taylor says, referring to a management evaluation technique invented by the Massachussets Institute of Technology's Robert Kaplan and David Norton in the late 1980s. The balanced scorecard considers a number of non-financial measures of success, such as learning and growth.

Despite its holistic, non-financial elements, the balanced scorecard, like its rivals, was developed to satisfy private sector IT managers who found using a financial indicator alone was of limited use. Even in a private sector organisation, Taylor says, you need to take account of more than just how you appear to shareholders. Staff training, internal business processes and customer perception must also be considered. Bernard Marr, from the Cranfield School of Management, says it is helpful for public sector organisations to modify the terms used to fit their own aims.

Marr makes bolder claims for the progress of the scorecard approach. "Almost every local authority has a balanced scorecard in place," he says. Among open adherents to the practice are the Ministry of Defence and Newcastle City Council.

Terry Dailey, of the ODPM's e-government Capacity Building Programme, is less certain about the inroads it has made. "Balanced scorecards are around, but are not very prevalent. It is not something to get people started with. It tends to be second stage," he says.

Only between one third and a half of councils are doing what Dailey would call "process mapping", a management technique using a type of flow chart where managers are encouraged to draw diagrams of the inputs, risks and business processes to help them think of possible improvements. But the disagreement about statistics is perhaps because Marr prefers a fairly broad definition of the balanced scorecard.

"It is simply a synonym for performance measurement and management strategy," he says. The problem, according to Marr, is that those who think they are using a balanced scorecard-influenced performance management system are not doing it properly. "A lot of people have read the first four publications on balanced scorecards, but have not moved on. But over the past 15 years scorecards have moved on."

According to Marr, those looking to use the technique need to ask themselves what their organisation or part of the organisation does, what resources it has to deliver, who the stakeholder is and what their intangibles are. These intangibles, he says, can be human capital, skills and IT capabilities.

From the point of view of an IT manager, it would be better if the organisation had asked itself the same set of questions first. This would allow the manager to develop a scorecard to deliver a resource supporting the scorecard-based strategy of the organisation as a whole.

The problem with a lot of public sector organisations is that the measurements they make are not allied to organisational goals, says Marr. "Instead of thinking about strategy they take the scorecard as a measurement tool. Public sector adopters will often simply work out which measurements they can make, often picking the easy ones. You end up with a useless measurement tool, with too many irrelevant measures."

A back-of-the-envelope job is not the best way to go about it, according to Marr. "To make it work well you need a lot of people to access it," he says. A performance management software package is the best way to achieve this, he believes. Software companies are scrambling for business from the 50% of local authorities which have expressed an interest. There are between 30 and 40 products on offer, which cost from £30,000 to £1m, depending on the size of implementation. These can be divided into three broad categories: those offered by enterprise resource management software companies such as SAP, Oracle, PeopleSoft and Hyperion; ones from business intelligence heritage such as Business Objects and Hyperion; and customised products from PerformanceSoft and Inphase.

Among a multitude of factors affecting the choice is an organisation's existing infrastructure. For example, Marr says, it makes perfect sense for an organisation with a SAP system to buy a SAP performance management system, but it may not be best for other organisations.

Terry Dailey is more sceptical about the benefits of performance management software earlu on. "Using Microsoft Project can, at early stages, hinder the process. It would be better to sometimes do things manually. If you do the simple things manually, you understand it properly," he says. In his estimation, only 2% to 3% of organisations are using more sophisticated project management packages such as Artimis or Niku.

With all this change in management approaches going on in government, one important set of stakeholders - the taxpayers - might have to alter their expectations. "If people set their service expectations by those set by the private sector, people will have to accept they need to invest in IT," Taylor says. But perhaps the most reliable way to measure whether those paying think it is a worthwhile investment will continue to be through the ballot box.

Read more on IT project management