The National Audit Office's final report on the NPfIT
was very different to earlier drafts, which criticised the
programme. Was it influenced along the way?
In the first few words on its website, public spending watchdog
the National Audit Office declares, "We are totally independent of
government."
But last year the NAO went on the defensive after receiving a
letter from Connecting for Health, which is running the NHS's
National Programme for IT (NPfIT), the UK's largest
computer-related investment, costing £12.4bn.
Under the Freedom of Information Act the NAO has released some
correspondence between one of its senior auditors, Chris Shapcott,
and Richard Granger, IT head of the NPfIT who is chief executive of
NHS Connecting for Health and also director general of NHS IT.
In his letter dated 17 March 2005, Granger shows an apparent
disapproval of the possibility that the NAO had been actively
engaging and encouraging third parties to examine the work of the
NPfIT.
Granger said to Shapcott, "When we last met you assured me that
whilst the NAO may have been in receipt of correspondence from
interested members of the public, you had not been soliciting
correspondence or actively engaging and encouraging third parties
to examine the work of the NPfIT."
The NAO wrote a deferential two-page reply which pre-dated by
more a year the final report of the audit office on the NPfIT - a
report that Conservative MP Greg Clark described as "gushing".
Drafts of the NAO's report on the NPfIT, which were also
released under the Freedom of Information Act, show that by the
time the final report was published some serious criticisms of the
programme by third parties had been removed or toned down.
Even the NAO's headline in draft reports of "Concerns raised in
correspondence with the National Audit Office" was changed in the
final report to the more deferential "Matters raised in
correspondence with the National Audit Office".
Convention dictates that the NAO cannot publish its studies into
the value for money of major projects such as the NPfIT until the
audited department and agency, in this case Connecting for Health,
agrees to sign off the final report.
In the case of the NAO's report on the NPfIT, it was delayed and
went through three drafts before publication of the final report,
which contained extra pages favourable to the NPfIT.
The NAO's reputation among some civil servants is that of a
fearsome watchdog with the power to investigate any programme or
project in almost any central government department.
If it wishes, it can commission third parties to do research on
its behalf. Its budgets are discussed by MPs on the House of
Commons Public Accounts Commission. But beyond agreeing its
publications with the audited departments, the NAO is accountable
to nobody on the wording, tone and findings of its reports.
One MP said, "Sometimes the NAO is fiercely independent and at
other times you get the feeling that civil servants are simply
auditing their colleagues."
So did the exchange of correspondence in 2005 presage the NAO's
final report on the NPfIT more a year later?
Granger wrote the letter after Connecting for Health received an
independent paper on the NPfIT. The paper was written by the UK
Computing Research Committee, which comprises an expert panel of
computing researchers from academia and industry who are members of
the Institution of Engineering and Technology and the British
Computer Society.
The health minister Lord Warner had received the committee's
paper and passed it to Connecting for Health. The paper raised some
awkward questions about the NPfIT, some of which have not been
answered even today.
The first question, for example, asked, "What are the top-level
objectives that the new systems are designed to achieve and which,
if not all met, would mean that the project had failed?"
Connecting for Health gained the impression that the NAO had
asked the UK Computing Research Committee to prepare the paper.
So Granger queried this with Shapcott. "When we last met you
assured me that whilst the NAO may have been in receipt of
correspondence from interested members of the public, you had not
been soliciting correspondence or actively engaging and encouraging
third parties to examine the work of the NPfIT," Granger wrote.
"I was therefore somewhat surprised to see the reference in the
attached paper from the UK Computing Research Committee, dated
September 2004, which reads as follows: 'These questions have been
prepared at the request of the National Audit Office to help them
review the NPfIT'."
In his reply Shapcott explained that Martyn Thomas, on behalf of
the UK Computing Research Committee, had approached the NAO with an
offer to help with its inquiry into the NPfIT.
"In view of the standing of both the committee and Dr Thomas
personally, we suggested a meeting to Dr Thomas to hear his views
the result is the paper which you enclosed with your letter," he
wrote to Granger.
Shapcott emphasised that the paper from the research committee
had not been commissioned by the NAO.
"I was particularly concerned at the possibility that you
[Granger] might feel that I had misled you in the assurance I gave
you that we had not commissioned an academic review of the
programme."
Shapcott said that the NAO wanted to carry out its examination
of the programme in a "co-operative, constructive and fair way" and
added, "I would be disappointed if the way this paper reached you
has given you the wrong impression."
But Shapcott also pointed out that it was "normal National Audit
Office practice to talk to a wide range of people when we are
conducting our examinations".
He said, "We are also aware that some of the people we are
speaking to may be biased or may not be in a position to make a
balanced assessment of performance.
"But we think it is important that we are well informed about
the views and interests of people involved with the programmes we
are examining
"I hope this reply reassures you but would be very happy to
discuss further if you wish."
The NAO's final report omitted a third-party survey by the
London School of Hygiene and Tropical Medicine on the NPfIT and a
report commissioned by the NAO into how the approach of the NPfIT
compared with the US and other countries. Neither report was
particularly flattering about the NPfIT.
Other criticisms of the NPfIT by the NAO, including examples of
the negative effect on trusts of the delays in the delivery of core
software, were removed during the six months that the draft reports
underwent a "clearance" process with Connecting for Health.
Computer Weekly asked Connecting for Health whether its letter
to Shapcott was confrontational and had put the NAO on the
defensive. We also asked if Connecting for Health had sought an
assurance from the NAO that it had not been soliciting
correspondence or actively engaging and encouraging third parties
to examine the work of the NPfIT.
Its spokesman replied, "The NAO undertakes clearance of its
reports to verify their accuracy. As is required, we contributed to
this routine process. The process that was followed regarding the
production of drafts and clearance was entirely conventional.
"The NAO report into the National Programme for IT involved a
lengthy and highly detailed study, which took numerous inputs from
a variety of quarters. It is entirely reasonable that senior
officials should check with the NAO the status of information and
inputs they are receiving given the volume and variety of
information collected."
It added, "The UK Computing Research Council's engagement with
the NAO is a matter for the NAO."
Warner said he is pleased with the final NAO report. So is
Connecting for Health, which has cited the final NAO report as an
endorsement of its work on the NPfIT.
But some may be left questioning whether the NAO's final report
on the NPfIT was as robustly independent as the audit office's
reputation. They may also ask why Connecting for Health seemed so
concerned about a third party review of the NPfIT.
The NAO is to publish a new report on the NPfIT.
Key questions experts asked about NHS's £12.4bn IT
programme
In 2005 the UK Computing Research Committee, which is made up of
leading computer experts, submitted a paper to the National Audit
Office, raising questions about the NHS's £12.4bn National
Programme for IT. The paper triggered a letter from Richard
Granger, IT head of the NPfIT, to the NAO.
These are some of the questions the committee suggested the NAO
ask in its review of the NPfIT. Even today, there are still no
clear answers to many of the issues raised by the committee.
- What are the top-level objectives that the new systems are
designed to achieve and which, if not all met, would mean that the
project had failed?
- Are there alternative solutions to the identified problems that
would have a better ratio of benefits to costs than the proposed IT
solutions?
- How widely will the proposed systems have to be deployed and
used before they can achieve their planned benefits at a level that
justifies the costs?
- How have you ensured that the stakeholders really understand
how the new systems will affect their working processes?
- How long could access to data be permitted to take before the
delays became unacceptable in the most time-critical of the
required functions?
- What estimates have been made of the cost and impact on NHS
services of designing new workflows and processes, and training
staff to use the new systems?
The paper was submitted to the NAO on behalf of the committee by
Martyn Thomas, a visiting professor in software engineering at
Oxford University, who is among 23 academics who are calling for an
independent technical audit of the NPfIT.
The committee comprises an expert panel of the British Computer
Society, the Institution of Electrical Engineers and the Council of
Professors and Heads of Computing. It was formed in November 2000
as a policy committee for computing research in the UK. Its members
are leading computing researchers from UK academia and
industry.
British Computer Society expresses concerns about the
NHS IT programme
The final report of the National Audit Office on the NHS’s
National Programme for IT (NPfIT) includes
few specific criticisms of the scheme – despite detailed concerns
expressed to auditors by third-party organisations.
Glyn Hayes, chair of the British Computer Society’s Health
Informatics Forum, for example, has disclosed that the BCS
contacted the NAO about what it regarded as the “monolithic central
National Spine”, which is being designed to hold medical records on
50 million patients in England.
Hayes wrote to Martyn Thomas, one of 23 academics who have
written an open letter to the House of Commons Health Committee,
calling for an independent technical audit of the NPfIT.
The BCS sent Computer Weekly a copy of Hayes’ letter to Thomas.
It stated, “We at BCS Health Informatics Forum have always been in
favour of such a technical review.
“There are aspects of the project, in particular the monolithic
central National Spine, which worry us greatly.
“This was one of the points we made strongly to the NAO.
“Obviously such a centralised system could be made to work
technically, although they are having major problems doing so
because of the scale and scope of the NHS.
“Our main concern being that a centralised system will not work
in the complex organisational structure of the NHS. A distributed
architecture would have been much more flexible.”
This specific concern of the BCS was not included in the NAO’s
report on the NPfIT.
The Health Committee is expected to hold an inquiry into the
programme.
Vote for your IT greats
Who have been the most influential people in IT in the past 40
years? The greatest organisations? The best hardware and software
technologies? As part of Computer Weekly’s 40th anniversary
celebrations, we are asking our readers who and what has really
made a difference?
Vote now at:
www.computerweekly.com/ITgreats