The public sector is spending £12.5bn on IT this year. Do central government IT projects fail more than private sector ones. If so, why? And can the Office of Government Commerce reverse the process? Julia Vowler reports
The recent war of words between senior government officials and software industry leaders over who is to blame for the UK government's dismal track record for IT projects has given a long-running debate a new spin.
Is e-envoy Andrew Pinder and chief executive of the Office of Government Commerce Peter Gershon justified in accusing the UK's software industry, or is Dennis Keeling of the Business Application Software Developers Association (Basda) right to blame the government for having a flawed procurement policy?
One thing is beyond doubt: the UK government is ploughing an unprecedented amount of money into IT. Public sector analyst firm Kable predicts that total IT spend for the public sector in 2003/2004 will increase by nearly 25% from last year, at just less than £12.5bn, with central government spending more than £3bn. Then there is the government's £2.3bn national IT programme for the NHS. Up to £3bn of government IT spending will go on electronic service delivery since all government services have to be available electronically by 2005. About a third of government IT spend goes on services - typically outsourcing and private finance initiative contracts.
But is that money mis-spent? Damning reports by the National Audit Office of government IT projects - such as the Libra system for the magistrates courts and the Passport Office that have overrun their budgets by millions of pounds - have become commonplace.
However, should those chilling figures be seen in the context of IT implementation in general? Do other sectors fare better? Some experts think not.
"In my experience the UK government's track record is not significantly worse than for IT overall," says Andrew Davies, who serves on a panel of advisers to the NAO and is visiting professor in information systems at Cranfield School of Management.
He argues that the government's IT track record looks worse than other sectors because it is more visible. "We know more about it. There's a whole programme of scrutiny that identifies and investigates IT failures."
Davies adds that the private sector is simply getting away with a poor track record - but out of the public view. "I have seen disasters on the same scale in the private sector as in the public sector," he says.
In defence of UK government IT, "They get an awful lot of things right, but those don't hit the headlines. The NAO is not interested in them," he says.
So does this mean that it is time to shrug shoulders and accept the IT disasters as par for the course? Viewed as money displaced from hospitals and schools, IT disasters are even less forgivable. Nor, unlike the private sector, can survival of the fittest lead to self-regulation via efficiency versus extinction.
But why do UK government IT projects go wrong?
The first reason is size. Very large IT projects are inherently difficult to do, and some government IT projects are extremely large, involving the population of the country. With such a vast scope, the multiplier effect of a national roll-out will ensure that any mistakes or scope-creeps are amplified massively. Moreover, when very large IT projects are implemented in a big-bang, one-off fashion after years of development, instead of in discrete chunks every six months, the likelihood of failure or overrun is greater. Even worse, the longer a project takes to deliver, the more likely it is that the initial specification will be changed, resulting in uncontrollable cost and time overruns.
"It used to be the case that projects which took more than three years were more likely to be cancelled than to go live," says IT parliamentary lobby group Eurim in a recent report.
Now, private sector projects which take more than three months are more likely to be canned before roll-out. And projects that take more than six months to plan and procure are also doomed.
The second reason behind wayward government IT projects comes in the commissioning stage. When laws change, the underlying IT that supports those changes may be substantial. Politicians do now consider the IT implications of legislation, although the danger is that they receive optimistic answers from civil servants and suppliers.
"In government departments the policy unit does not always think too carefully about the impact of IT," says Davies. "And in NAO reports it is pointed out that decisions are taken without [adequate awareness] of the implications for IT."
Thirdly, government IT lacks the appreciation that there is no such thing as a pure IT project, but only a business project with a strong IT component.
These factors, however, are not unique to public sector IT. Indeed, Keeling and Davies, among others, would argue that government projects are no larger than the largest in the private sector - such as banks, who also deal in millions of people, as well as multimillions of pounds of IT spend.
But where the government does hit a problem not shared by the private sector is the issue of civil service IT management. Central government mandarins are remote from their user communities - unlike, says Kate Mountain, chief executive of local authority IT managers' association Socitm, council staff. She argues that local authorities are far more successful at cost-effective IT implementation.
"We are closer to the people who will use our systems," she says, "and closer to the politicians [who commission them] - there is immediate accountability. In central government civil servants change jobs regularly and teams are never stable."
The culture of the civil service is to win promotion by moving post, observers say, which militates against continuity of project management or to the accumulation of skills necessary for ensuring successful IT implementation.
This "lack of top management commitment and involvement", says the Eurim report, can be exacerbated by "lack of experience in how to get value for money as a customer or in how to manage the associated change programme. This is the single most important cause" of government IT project failure.
There is also, argue government critics, a damning lack of adequate risk management when it comes to spending taxpayer's money on IT. This is allied to unrealistic over-ambition, says Eurim.
Keeling agrees, "Risk assessment is not there. For example, the Inland Revenue is proposing to change the National Insurance numbering system in the same year as it is [reletting] its outsourcing contract."
An essential part of risk management is managing uncertainty. This is not something government IT is traditionally good at, says Davies. He says the government needs to be more flexible in its procurement process.
This means, of course, that suppliers must be more flexible too, contractually and technically, embracing open systems and not being focused on short-term profits and proprietary systems, says Eurim's report.
But does government need to rely so heavily on bespoke systems, however much they are broken down into manageable, bite-size deliverable pieces? The lack of exploitation of packaged software is a major reason for why government IT is so expensive, says Keeling.
"Not everything could be package software, but I would say 80% could be," he says. "At the moment it's more like 20%."
For Keeling it is a matter of educating the public sector just as the software package industry had to educate the private sector that it was unnecessary to constantly recode systems already existing in the market.
"Some years ago the Benefits Agency was proposing to write a financial ledger system for £5m because it thought no package system would be large enough to handle the volumes," recalls Keeling. "We found them a mainframe package for less than £1m."
Bruce Hudson of Meta Group agrees, "Package software is the way to go [for government IT], though the culture has resisted it. Over five years a bespoke system would be, on average, about five to seven times more expensive than a package."
Not everyone is as enthusiastic, however. "It is not an environment that packages can ever make a huge impact on," argues Davies. "Some systems are extremely specific, and the requirements are set by law. It would be far more difficult to adapt them."
The main government body responsible for improving the success rate of IT projects is the Office of Government Commerce.
However, the OGC is treading a delicate line, keen to improve IT performance by encouragement rather than punishment. Those in favour of this strategy would argue that the latter is counter-productive.
"The investigatory and scrutinising process is deeply irritating to all who suffer it, especially the negative function of the House of Commons Public Accounts Committee," says Davies.
Even Eurim's damning government IT report, which spotted large numbers of causes for systemic failure, says, "It is vital to change the current negative role of the Public Accounts Committee if we are to change the culture and attitudes to risk of public servants."
Gershon is keen to point out that not all government IT is an automatic disaster. "The Customs & Excise Import & Export system, Outsourcing for National Savings and Investments, and NHS Direct have been major successes," he says.
But without the threat of public exposure, and being demoted or sacked for a series of botched IT projects, will government IT projects ever get any better?
The only true test of the effectiveness of the OGC in improving government IT will come if the NAO reports on IT projects that have been scrutinised by the OGC start to show thumbs-up judgements.
But the impact will not be immediate. The OGC has been granted no retrospective powers to subject projects started prior to February 2001 to Gateway reviews. It should also be remembered that the OGC only has limited resources - about 700 consultants from Whitehall departments and the private sector.
In the meantime, expect more buck-passing and rows over delayed and scrapped IT projects.
Can the OGC turn UK government computing around?
The Office of Government Commerce is chaired by Peter Gershon, whose previous employer was Marconi in the private sector. The agency was set up three years ago to be an independent source of expertise, advice and monitoring of government procurement projects - including IT.
It was given the task of taking out £1bn of spending across the board, by "buying smarter" but without cutting projects or services. The OGC claims it has "comfortably exceeded" that target, and is now aiming to cut £3bn in the next three years. How much comes from IT will not be known until the National Audit Office makes its report this summer. The OGC is tackling the government IT problem in at least three ways.
It has introduced aggregated purchasing, so that departments can get better economies of scale and more transparent pricing. However, critics claim aggregated purchasing may do more harm than good as it sidesteps European Commission open tendering laws.
In February a deal was done with Oracle to "secure an average 11% reduction in the cost of technology products directly and indirectly purchased from Oracle. With transparency and standardisation built-in, departments will be able to secure the best possible prices," says a spokesperson for the OGC.
The agency is also tackling the assumption that government IT as the preserve of a handful of large suppliers, as these are the only ones able to tackle large government IT procurement, by introducing more competition. E-procurement is, according to the OGC, making it easier and cheaper for smaller suppliers to bid for government work.
Framework contracts also allow the selection of a "basket of suppliers" that departments can use. This increases choice as does breaking up contracts to select 'best of breed' suppliers for each component, rather than a single supplier which may be good for some aspects of the project but not others.
The OGC has also set up a Senior IT Forum for IT purchasers and suppliers to improve behaviour on both sides. "Are suppliers promising more than they can deliver? We want this to stop," says Gershon.
Have these efforts improved government IT procurement? "We are seeing the beginning," says the OGC spokesperson cautiously. Changes in both supplier and purchaser behaviour "won't be overnight".
The OGC can act as an overall programme management office by instigating centres of excellence in project management - including vital risk management skills - and encouraging the take-up of "lessons learnt" via its projects database. The agency recommends that every project has a Senior Responsible Owner - a grade 5 civil servant or above, right up to permanent secretary level (the civil servant who heads a department) - who owns the budget and the deadline. In this way the OGC wants to tackle the endemic problem of lack of project accountability. "Their name is on the project, they can't hide and they can be called to account. We want to encourage the SROs - and the whole project team - to see the project through to the end," says the spokesperson. "If a new SRO has to come in we'd meet up and gauge just how up to speed he was."
Gateway review This is where the real teeth of the OGC bite. Since February 2001 every government IT project has been subject to mandatory review, right from assessing the feasibility of goals through six stages to completion and delivery of benefits.
If, at any stage, a project is deemed by the review to be going off the rails, "we can red-light it", says the OGC. "We have the power to recommend halting the project to the permanent secretary. He can overrule us, but if the project then goes pear-shaped it will have his name all over it and he'll be explaining that to the NAO. "It's rare, if ever, that we are overruled."
However, the OGC is careful to stress that "at the end of the day it is down to a department how it spends its money. If we could mandatorily stop a project and take power over it from the permanent secretary, then that could cause concern in the departments", the agency says.
Has it got all the powers it needs? "We are happy with the current set-up," says the OGC spokesperson. It also stresses that the results of Gateway reviews are confidential. There is no public disclosure as to which, if any, current IT projects have been held at yellow or red alert. "We don't want to be seen as project police."
Naming and shaming is out - at least until the NAO report comes along.