It did sound bad, when the Public Accounts Committee report on NHS IT summed it up this week.
The NHS National Programme for IT will not only have cost £10bn by the end of life of its systems in 2024, according to a National Audit Office review that informed the committee report. A lot of it remained undelivered. The NHS had derived just £2.7bn financial benefit out of £6.4bn spent on NPfIT between late 2003 and March 2011.
These numbers loomed in the media like a dreaded borak – like the abomination NHS IT is perceived to be. No matter that the NAO review – written under instruction from the committee – had to make do with flimsy numbers. Just saying the words £10bn and NHS IT is enough to incite hatred and get headlines, perhaps the other way around.
“Cost of abandoned NHS IT system hits £10bn,” said the papers.
The coalition government shelved the scheme two years ago but it’s still wasting money!
“The taxpayer is continuing to pay the price for the ill-fated National Programme for IT in the NHS,” said Richard Bacon, who has led the scandal over NHS software as the Public Accounts Committee’s glitchfinder general.
But this latest condemnation of NHS IT was tired and grubby.
It was true that the Department of Health calculated the NPfIT bill would be £9.8bn by 2024. But that was half only half the story.
The number came from a cost/benefit calculation. The other half of the story – the benefit – was £10.7bn by 2014, more than the cost. Bacon would have been more justified to say the taxpayer continued to reap the benefits from NPfIT.
It even seemed like a decent price, £10bn – even after all the software glitches that have beset the department, and the Übermenschen day-rates enjoyed by private sector IT consultants.
Chris Yapp, an IT consultant, has estimated that programme-related IT cost the NHS about £1bn-a-year when NPfIT was conceived in 2002. Think of inflation since then, and how organisations’ needs for computing have inflated too. Now look at the NAO numbers again.
The department estimated NPfIT cost was £7.3bn between late 2003 and March 2012 – roughly £900,000-a-year. NPfIT’s lifetime cost, to 2014, is roughly £500,000-a-year.
The cost/benefits calculations were meaningless anyway. They were an accounting tautology. They said that by the programme’s end of life, when it had delivered its systems – when it had achieved its aims – it would have realised all its benefits.
The committee didn’t need the NAO to tell it that. But it did need the NAO to say it was difficult to measure the financial benefit at all.
Financial benefit – the financial return on investment – the ROI: the auditor viewed the numbers with “very considerable uncertainty”.
Since when do you measure the return of your investment in a health system in pounds and pence? The department never set an ROI target for its NPfIT investment, said the NAO. NHS organisations never sought to measure it. An apparently sizeable proportion of 4,715 NHS organisations involved in the programme nevertheless pulled together to do the calculations.
They did time and motion studies to try and measure the ROI. They did staff surveys. They struggled. The resulting data was inconsistent and unreliable. The ROI was immeasurable. The NAO said some of investments had no measurable financial benefit at all. Infrastructure costs, it said, had no measurable benefit.
It’s not that you wouldn’t claim there was no benefit in laying computer infrastructure. You just couldn’t measure it in pounds and pence, apparently.
“For some elements only qualitative benefits were identified,” said the NAO.
Still, they declared NPfIT had delivered ROI of £2.7bn between 2003 and 2011, against £6.4bn costs.
Those benefits that were immeasurable may have been of more interest – they were invaluable. Yet the NAO neglected to break the cost and benefit figures down to show what derived from different parts of the programme.
So it was impossible to verify what seemed the obvious flaw in the exercise. The costs and benefits could not be meaningfully compared because they measured different things. The costs were comprised mostly with things that had been delivered but had no measurable benefit. The benefits meanwhile comprised mostly a forecast of things not yet delivered.
While infrastructure and similar costs accounted for most of the £6.4bn cost to date, in other words, they had no impact on ROI because their benefits were immeasurable. The cost/benefit ratio was immeasurable.
The NAO used words to convey how much more there was to the programme than a flimsy ROI sum. It had already delivered most of a national network, medical image archiving and despatch, email, the choose and book appointments system, and what is known as the spine.
Blurb by BT, supplier of this backbone, reckons it has 899,000 users, processes 150m transactions-a-month, recording births, deaths, and GP registrations, processing 675,000 prescriptions-a-day, making 400,000 security authentications, 39,000 hospital appointments, and administering NHS business intelligence, management reporting, processing two billions queries-a-month on what it says is one of the world’s largest data warehouses. It also administers Payment by Results – an NHS accounting system – processing £35bn transactions-a-year.
So this is what the NHS got for £0.9m-a-year between 2003 and 2011. What was the benefit?
Someone might have told the public accounts committee there was little benefit in a b enefits assessment that couldn’t measure the benefits.
The NAO said something along those lines in the report the committee instructed it to write on the matter.
“There is very considerable uncertainty around the benefits figures reported in the benefits statement,” it said.
Most of its uncertainty derived from those systems that remained undelivered: local patient records systems, the national summary patient record system and electronic prescriptions.
Unlike infrastructure, these systems would cost least but deliver most measurable ROI. That much could be deduced from the NAO numbers, where the absence of these components could be seen: in a small gap between present total and final estimated costs, and a large gap in present and final estimated ROI.
The NHS had yet to realise 98 per cent of benefits for these systems, said the NAO, because they had not been delivered.
It was one of those statistics that is so audacious in its absurdity that it transcends comprehension, leaving befuddled, yobbish alarm. They aint only aint delivered the systems but they aint got 98 per cent of the benefits of wot they aint delivered neever. The statistic is more damning than the fact.
The opportunity cost of the these undelivered systems – the real tragedy of NHS IT – may be so great it seems immeasurable: ten years waiting on BT and Computer Sciences Corporation (CSC – primarily a defence contractor) to deliver the next generation of health IT software when everyone could have been getting on without them.
“Have you been to a hospital lately,” Tola Sargeant, research director at TechMarketView asked Computer Weekly this week.
“It’s scary. They’re still pushing around trolleys of paper records.”
The opportunity cost/benefit can be measured. And it may be a more fruitful line of enquiry than the one that gives us the politically potent but fairly meaningless £10bn, 20-year cost.
It would require a functional measure of benefits derived from a public service computer system.
Take the undelivered NHS programme software. CSC (the defence contractor) had implemented interim systems where hospitals needed it, while they waited for Lorenzo, its next-generation health software that was overdue in 2005. The NHS paid CSC £995m since 2003 for 2,665 interim systems, delivered to 179 NHS trusts and 1,800 GP surgeries.
While the NHS was denied 98 per cent of the benefit promised by the unfinished Lorenzo, it benefited instead from alternative implementations of 87 patient administration systems, 128 child health systems, 38 systems for operating theatres, 29 for accident & emergency units, 130 for community practices, 80 for prisons and six ambulance systems.
The NAO said its survey did factor the £benefit of interim systems in its calculations. Those were the systems the NHS used for a decade while waiting for NPfIT – systems NPfIT suppliers delivered in lieu of the finished goods.
These may have measured poorly against the final system: the fully-specified finished goods. The final £benefit measures only what NPfIT promises to deliver in 2024 against what has been done. It has not measured what has been done against what is presently feasible, the NAO told Computer Weekly.
Are there systems anywhere in the world, in other words, that could be installed now to give the NHS what is promised for 2024? Or is 2024 more a measure of what the health software industry can do than what the NPfIT contract can deliver?
A creditable assessment of an IT programme would take all these things into account. Not in the way an accountant or a lawyer might compare a sheet of functional requirements drawn up in 2002 to a list of software functions delivered in 2013. But by a measure of functional benefits of one set of alternatives to another.
So now Lorenzo and the rest of the undelivered software is being delivered, just how good is it, and how much progress could the NHS, and the health software industry more generally have made without it? Where is the rest of the industry, functionally? What was the benefit? Not the £benefit. The benefit.
The accounts committee and the public auditor it instructs may not be geared up for this sort of assessment.
Even what they do never gets to the point of why, for example, it costs a hospital about £0.9m-a-year for the next generation Cerner patient software being delivered by BT.
It is paint-by-numbers accountancy – cretinous number crunching: you said it would cost this much and take this long, but its taken a bit longer and cost a bit more – (it’s an outrage!, roar the glitchfinders).
Auditor and committee never get to the point because they aren’t allowed to look at the numbers: suppliers like BT and CSC insist it’s “commercial in confidence”.
So they clod-hop over the big ticket numbers, making big political stink bombs out of them, while the unanswered questions about invoice line items hang in the air, providing cover for political opportunists.
We have for example been encouraged to assume £0.9m is an outrageous sum for a BT patient system. It probably is. But what is it really worth? A functional £cost/()benefit break-down of a system might provide a more meaningful measure of its worth. An effort/benefit might be more meaningful still.
Other ways of measuring benefit – such as cost per user and cost per transaction – might be combined with a weighting of security demands, a measure of change, a consideration of pioneering efforts, a rate of software re-use, time and resource, to give a more useful assessment of cost/benefit.
The debate about public IT has been led by accountants. Yet the accountants have not even attempted to address the most fundamental cost-benefit-function questions in public computing, such as whether open source development would deliver a better – more capable – batch of functions per penny than farming the work out to a product supplier like BT or CSC.
We are led to believe that the NHS is stuck in the 50s thanks to NPfIT – that its software evolution has been stunted and what it will get will be backwards.
But Lorenzo is revered as the next generation in health software. They call it the SAP of healthcare. Its ambitions were audacious. Sargeant says it had never been done before. It may represent the evolutionary heights of an industry, not just all that the NHS could d o in 10 years.
It is safe to assume the health software industry progressed in disregard of any desire of the legal or accounting professions for the NHS to put its computing needs in aspic – in contract – in 2002.
The computing industry did not hold its own progress in stasis all the while. It did not stay stuck in the half-lit days before NPfIT, before CSC bought iSoft and before System C – another important supplier of next generation hospital patient systems to the NHS – was bought by McKesson, the $122bn US medical supplies distributor (and another large US defense contractor); the days before health software became a global business.
It is tempting now to say the coalition vision for NHS IT is itself in stasis: stuck in a romantic, pre-NPfIT rural English idyll of bicycle baskets and antique tea shops: a hollow promise to win votes.
The coalition evoked this vision by seeking to cut CSC’s contract scope after it came to power in 2010.
CSC had been contracted £3.1bn to deliver 220 systems to hospitals and other NHS trusts. After (ongoing) renegotiations with the coalition government, its obligation has been cut to just 22 hospitals and £2.2bn.
This has been cause for more bovine outrage: its allotted systems were cut nine tenths, but its pay by only a third – surely no more evidence was required to show CSC continues to fleece the NHS, and the NHS continues to pour money down the drain?
Yet most of the £2.2bn is for work already done. Three quarters of it was for non-Lorenzo work, including the £995m for interim systems at 179 trusts and 1,800 GP surgeries. The other 19 trusts are presumably not in need.
The deal has nevertheless created the possibility that the sort of world the coalition envisaged for NHS IT might still evolve out of NPfIT: where the fabled hospital basement industry of do-it-yourself clinical software and small, innovative suppliers might unite to produce an open source software ecosystem that is the only viable alternative to systems developed by the global corporations, by US defense contractors.
It was something like the vision outgoing NHS chief executive David Nicholson relayed wistfully before the Public Accounts Committee in June.
“It was a basic patient administration system. But many clinicians created their own systems and connected it to the base system. So over the years we built up this extraordinary conglomeration of systems.
“The National Programme for IT offered to take it all away and replace it with something standard, which will probably be less than you have now. People go, ‘Why on earth would we do that?’. The idea of ruthless standardisation – that you can enforce a set of things on the NHS – has proved illusory.”
This romantic vision was the crucible of the coalition IT strategy for the NHS. Now the department has put up £500m to fund post-NPfIT systems development, it promises that this is what we will get: not a big wad of cash going to CSC, but a big wad of cash funding local innovation in NHS IT.
It seems, however, the result will be more akin to NPfIT than anything else we might have been told we may hope or imagine.
The department promised £500m over five years for patient and prescription systems on the condition that trusts matched it: £1bn in all, equal to what a department spokeswoman said it cut from the NHS contract with CSC.
Cut on the back of an envelope, at a run-rate of £1m per-hospital per year, that £1bn would cover about as many hospitals as have been cut from the CSC deal as well: 200.
In cost benefit terms the coalition may have achieved nothing. In terms of time to delivery it may make little difference now whether trusts implement Lorenzo or not. What will matter to the NHS is the functional benefit of its computing choices, and the £cost/()benefit not only of contracts but of software rights, ecosystems and development models.
It looks now like the coalition may have thrown away an opportunity to build an open source health software ecosystem – one that empowers through unity of purpose and endeavour, and by distributing the means of production, levels it into a collaborative, national venture – a bit like the NHS, only concerned with producing health software for the health system.
This is what the coalition promised for NHS IT. In terms of final ()benefit, it means pooling all efforts by all producers of all systems into the same family of software products. So instead of different suppliers re-£-inventing the wheel decade after decade (a situation described by industry veteran Philip Virgo in the first comment on this article), you get them making enhancements to what was already done. Collaboration trumps competition.
The coalition is instead delivering a market democracy, where software companies have the power, the rights, the spoils and, as healthcare is automated, the future. Without any concerted effort to the contrary from the department to nurture the alternative the coalition evoked when it wanted to get elected, many hospitals will simply contract with BT, CSC and McKesson.
The result will be something very similar to the National Programme for IT: expenditure on systems produced by the major software suppliers, perhaps with some exceptions, at about the same price in about the same time.
The committee smartly dismissed the coalition’s broader claim that it was “dismantling” NPfIT. Aside from renegotiating CSC’s contract, it had merely assigned each strand of the other 90 per cent of the £10bn programme to independent directors. Remember this the next time you hear the health secretary scoring political points over the National Programme for IT.