The National Audit Office report into the National
Programme for IT in the NHS has, for reasons that are unclear,
omitted to mention some of the most challenging aspects of the
programme.
Some important subjects are mentioned in passing or not explored
by the report, the contents of which were the product of prolonged
negotiation between the National Audit Office and the Department of
Health.
Questions not fully covered by the 60-page report range from the
aspects of the cost and scope of the project to systems delivery
and oversight of the project’s progress.
Cost and scope
The National Audit Office’s report makes no mention that a
Downing Street meeting, chaired by the prime minister Tony Blair in
February 2002, gave the provisional go-ahead for an NHS national IT
programme that would last two years and nine months. No mention was
made of how the NPfIT came to be a programme lasting at least 10
years.
Other key issues that the National Audit Office report did not
mention include:
● A “Project Profile Model” completed by health officials in
2002 to assess the risks of the NPfIT, which scored the programme
at 53 out of a maximum possible 72 and put the total whole-life
project costs at £5bn. Any score of 40 or more represents a
high-risk project. The National Audit Office report’s estimate for
the total whole-life project cost is now between £12.4bn and
£12.6bn.
● The business justification for the NPfIT was not produced in
draft form until the summer of 2003, a year after the programme had
successfully passed a Gateway review – an independent assessment by
the Office of Government Commerce of the project’s feasibility.
● How much money the Treasury has committed to the programme so
far and whether there is certainty that the remainder of the funds
will be made available.
● Whether the £12bn estimated cost of the programme represents
value for money, or whether the hundreds of millions of pounds
spent so far represent value for money.
● Whether a potential £1.5bn for central management costs is
reasonable.
● The costs to NHS trusts of the repeated deferrals and
cancellations of dates for going live with new nationally-bought
systems. In many cases old contracts have had to be renewed,
sometimes at a higher cost than before, or interim systems
acquired, staff stood down, planning time wasted, and training
repeated.
● There is concern among NHS trusts over the affordability of
the programme in general over the medium or long term. Connecting
for Health has itself said that the NHS funding shortfalls are a
“hot topic”. But the report reflects only the concern within NHS
trusts about whether they can afford new x-ray technology – picture
archiving and communications systems. It does not mention
uncertainty about the local funding for the national programme in
general.
● The extent of financial commitments by the NHS to suppliers,
or the level of fines faced by the NHS if it breaches the
contractual promises made on its behalf by the Department of
Health.
● Whether the NHS has paid any fines to suppliers or whether any
penalties are due to be paid.
● Suppliers may be paid most of the contract price even if NHS
staff do not use the new systems. The National Audit Office report
commends Connecting for Health for ensuring that up to 15% of the
supplier’s charges are based on usage. But it does not mention
whether the supplier can be paid up to 85% of the contract’s value
if the systems are not used.
● The cost of compensation paid to supplier EDS for cancelling
its contract to provide e-mail services, or whether the replacement
service from supplier Cable & Wireless has attracted many more
active NHS users than is likely to have been the case by now if
EDS’ contract had not been cancelled.
● Why exactly BT was paid an extra £69m for broadband “N3”
connections, why this was not included in the original contract,
and why a further £90m is necessary for integration and why this
was not included in original contract.
Oversight
Any major IT project needs thorough oversight. Its progress
should be transparent to the sponsors of the project and, if new
systems are to be adopted enthusiastically, to the eventual
end-users of what is supplied.
The National Audit Office report is part of that process, but it
does not refer to important studies that could allow others to
evaluate the project’s progress. Missing from the report are:
● The results of Gateway reviews by the Office of Government
Commerce. The NPfIT has been the subject of several Gateway
reviews, which assess how a project or programme is progressing at
various stages. In the past, the National Audit Office has reported
details from Gateway reviews on, for example, IT systems to support
the Criminal Records Bureau and tax credits.
● The results of independent reviews commissioned by the
government, such as a report on the NPfIT by management consultancy
McKinsey.
● Registers of the NPfIT’s risks and how each risk is assessed
to reveal the effect on the future of the programme should the risk
become fact.
Delivery
The audit office reported on the successful deployment of some
core parts of the NPfIT and noted delays to others, including the
integrated NHS Care Records Service, and slow take up of the
politically sensitive Choose and Book system. The report does not
mention:
● Most of the 261,983 bookings of hospital appointments using a
Choose and Book system up to
3 April 2006 were made on the telephone, in many cases because GPs
were unable to arrange bookings online.
● A shift away from national or “strategic” systems towards
“tactical” or standalone technology in local areas, in part because
of the difficulties of providing a country-wide networked system.
This move is reflected in the board papers of NHS trusts but not in
the report of the National Audit Office.
● The difficulties of an implementation at Nuffield Orthopaedic
Centre – although audit office staff visited the hospital earlier
this year. The trust was the first in the south of England to go
live in a roll-out of a basic version of the national programme’s
Care Records Service – a version which does not share records with
other hospitals across England.
The implementation led to some operations on patients being
postponed. The go-live has also led to changes in the
implementation plans for the south of England, in part to allow for
extra testing. In May, the trust’s director of nursing and
operations said Nuffield in some cases had been unable for nearly
four months to “identify the patients who need to be brought
forward for treatment within national access timescales”.
● The danger of systems for electronic health records being
installed but not being used – an issue raised by government health
officials in the US.
● Whether the long-term aim of a national care records service,
the main component being a shared electronic health record, is
likely to materialise.
Buy-in
The National Audit Office report pointed to criticisms by some
clinicians and other NHS staff of the NPfIT, but there are
important omissions in what it said. For example, there is no
mention of:
● The existence and work of the National Clinical Advisory
Board, which was disbanded after the premature departure of Peter
Hutton its chairman. Hutton and the board were in part responsible
for ensuring that the NPfIT was what clinicians wanted and needed,
but were critical of aspects of the programme.
● A “key message” from a Mori survey of NHS staff commissioned by
the NPfIT. It found that those interviewed, including doctors, “are
much more favourable towards the future goals of the programme than
they are towards the programme in its current shape”.
The audit office said it may do another report on the NPfIT in
the future.
The challenge of objectivity
The National Audit Office received a plethora of letters, papers
and other documents on the NHS’s National Programme for IT (NPfIT),
some of them containing strong criticisms of aspects of the
scheme.
But few of the criticisms have made their way to the main body
of the National Audit Office’s report on the programme. The report
is generally positive about the management of the NPfIT, and the
criticisms of those who submitted information have been diluted and
summarised in a generalised form in an appendix at the back of the
audit office’s report.
Those who provided information to the National Audit Office
included members of the British Computer Society, the Worshipful
Company of Information Technologists and people who have
represented organisations set up by Connecting for Health, which
runs the NPfIT.
In understanding what is important, the National Audit Office
staff are likely to have been helped by having placed one of their
senior executives as an observer on the programme board of
Connecting for Health for two years. But this has its pros and
cons. The placement gives auditors a valuable insight into the
programme’s vicissitudes. It also allows the audit office to give
Connecting for Heath’s executives an insight into the lessons from
other government IT projects.
But it leaves the National Audit Office vulnerable to a
perception of a potential conflict of interest. Could the National
Audit Office criticise a programme that has been advised by one of
its senior executives, even if he did not take part in decision
making?
In the end, the audit office has published a report which
contains less criticism of the programme than some speeches and
media interviews by Richard Granger, chief executive of Connecting
for Health. Where the report does criticise, its comments are mild
or they tend to be counterposed by a positive statement. In
contrast, the praise is effusive.
One of the biggest difficulties for the National Audit Office
has been reporting on a project that is not yet half way through
its 10-year life.
Usually the National Audit Office conducts post mortems on
projects that have ended, for good or ill. To what extent do
auditors want to criticise a project that is continuing, and in
which the support of the stakeholders is critically important to
success?
Health officials in Whitehall are sensitive to criticism, in
part because they do not want to feed widespread scepticism over
the work carried out so far on the NPfIT. And auditors, by writing
an uncritical report, cannot be accused of fuelling scepticism
among clinicians.
But if the National Audit Office writes an unbalanced report,
which reflects the strengths of a programme and few of its
weaknesses, is it doing its job well?
More information:
www.computerweekly.com/naoreport
Lessons from history
The National Audit Office has produced many reports into large,
expensive government IT project failures and its work has fuelled a
number of reports and best practice guidelines that are used today
to improve the delivery of government projects.
Its report on the National Programme for IT in the NHS does not
link the lessons learnt from earlier inquiries to what is happening
on the NPfIT. Examples include:
● A recommendation on the importance of aligning working
practices to new systems. This was a central lesson of a report on
the failure of Libra, a project to provide a national case
management system to magistrates courts. No mention is made in the
audit office’s NPfIT report of the fact that contracts were signed,
and software written, without agreement among doctors over what
changes doctors and nurses would need to make to their working
practices to make best use of the new systems.
● A report of the Cabinet Office in 2000 which recommended that
a project or programme’s senior responsible owner “should remain in
place throughout, or change only when a distinct phase of benefit
delivery has been completed”. The report on the NPfIT lists the
changes of senior responsible owners but does not say this
conflicts with good practice as set out by the Cabinet Office. Nor
does the audit office report call on the NHS to settle on one
senior responsible owner from now on.
● The National Audit Office’s report mentioned that local
procurements have represented poor value for money – but it made no
mention of the failure of national systems such as Read Codes 3,
and Hospital Information Support Systems, both of which the audit
office investigated in the 1990s.