Was NAO report truly independent?

The National Audit Office's final report on the NPfIT was very different to earlier drafts, which criticised the programme. Was it influenced along the way?

The National Audit Office's final report on the NPfIT was very different to earlier drafts, which criticised the programme. Was it influenced along the way?

In the first few words on its website, public spending watchdog the National Audit Office declares, "We are totally independent of government."

But last year the NAO went on the defensive after receiving a letter from Connecting for Health, which is running the NHS's National Programme for IT (NPfIT), the UK's largest computer-related investment, costing £12.4bn.

Under the Freedom of Information Act the NAO has released some correspondence between one of its senior auditors, Chris Shapcott, and Richard Granger, IT head of the NPfIT who is chief executive of NHS Connecting for Health and also director general of NHS IT.

In his letter dated 17 March 2005, Granger shows an apparent disapproval of the possibility that the NAO had been actively engaging and encouraging third parties to examine the work of the NPfIT.

Granger said to Shapcott, "When we last met you assured me that whilst the NAO may have been in receipt of correspondence from interested members of the public, you had not been soliciting correspondence or actively engaging and encouraging third parties to examine the work of the NPfIT."

The NAO wrote a deferential two-page reply which pre-dated by more a year the final report of the audit office on the NPfIT - a report that Conservative MP Greg Clark described as "gushing".

Drafts of the NAO's report on the NPfIT, which were also released under the Freedom of Information Act, show that by the time the final report was published some serious criticisms of the programme by third parties had been removed or toned down.

Even the NAO's headline in draft reports of "Concerns raised in correspondence with the National Audit Office" was changed in the final report to the more deferential "Matters raised in correspondence with the National Audit Office".

Convention dictates that the NAO cannot publish its studies into the value for money of major projects such as the NPfIT until the audited department and agency, in this case Connecting for Health, agrees to sign off the final report.

In the case of the NAO's report on the NPfIT, it was delayed and went through three drafts before publication of the final report, which contained extra pages favourable to the NPfIT.

The NAO's reputation among some civil servants is that of a fearsome watchdog with the power to investigate any programme or project in almost any central government department.

If it wishes, it can commission third parties to do research on its behalf. Its budgets are discussed by MPs on the House of Commons Public Accounts Commission. But beyond agreeing its publications with the audited departments, the NAO is accountable to nobody on the wording, tone and findings of its reports.

One MP said, "Sometimes the NAO is fiercely independent and at other times you get the feeling that civil servants are simply auditing their colleagues."

So did the exchange of correspondence in 2005 presage the NAO's final report on the NPfIT more a year later?

Granger wrote the letter after Connecting for Health received an independent paper on the NPfIT. The paper was written by the UK Computing Research Committee, which comprises an expert panel of computing researchers from academia and industry who are members of the Institution of Engineering and Technology and the British Computer Society.

The health minister Lord Warner had received the committee's paper and passed it to Connecting for Health. The paper raised some awkward questions about the NPfIT, some of which have not been answered even today.

The first question, for example, asked, "What are the top-level objectives that the new systems are designed to achieve and which, if not all met, would mean that the project had failed?"

Connecting for Health gained the impression that the NAO had asked the UK Computing Research Committee to prepare the paper.

So Granger queried this with Shapcott. "When we last met you assured me that whilst the NAO may have been in receipt of correspondence from interested members of the public, you had not been soliciting correspondence or actively engaging and encouraging third parties to examine the work of the NPfIT," Granger wrote.

"I was therefore somewhat surprised to see the reference in the attached paper from the UK Computing Research Committee, dated September 2004, which reads as follows: 'These questions have been prepared at the request of the National Audit Office to help them review the NPfIT'."

In his reply Shapcott explained that Martyn Thomas, on behalf of the UK Computing Research Committee, had approached the NAO with an offer to help with its inquiry into the NPfIT.

"In view of the standing of both the committee and Dr Thomas personally, we suggested a meeting to Dr Thomas to hear his views the result is the paper which you enclosed with your letter," he wrote to Granger.

Shapcott emphasised that the paper from the research committee had not been commissioned by the NAO.

"I was particularly concerned at the possibility that you [Granger] might feel that I had misled you in the assurance I gave you that we had not commissioned an academic review of the programme."

Shapcott said that the NAO wanted to carry out its examination of the programme in a "co-operative, constructive and fair way" and added, "I would be disappointed if the way this paper reached you has given you the wrong impression."

But Shapcott also pointed out that it was "normal National Audit Office practice to talk to a wide range of people when we are conducting our examinations".

He said, "We are also aware that some of the people we are speaking to may be biased or may not be in a position to make a balanced assessment of performance.

"But we think it is important that we are well informed about the views and interests of people involved with the programmes we are examining

"I hope this reply reassures you but would be very happy to discuss further if you wish."

The NAO's final report omitted a third-party survey by the London School of Hygiene and Tropical Medicine on the NPfIT and a report commissioned by the NAO into how the approach of the NPfIT compared with the US and other countries. Neither report was particularly flattering about the NPfIT.

Other criticisms of the NPfIT by the NAO, including examples of the negative effect on trusts of the delays in the delivery of core software, were removed during the six months that the draft reports underwent a "clearance" process with Connecting for Health.

Computer Weekly asked Connecting for Health whether its letter to Shapcott was confrontational and had put the NAO on the defensive. We also asked if Connecting for Health had sought an assurance from the NAO that it had not been soliciting correspondence or actively engaging and encouraging third parties to examine the work of the NPfIT.

Its spokesman replied, "The NAO undertakes clearance of its reports to verify their accuracy. As is required, we contributed to this routine process. The process that was followed regarding the production of drafts and clearance was entirely conventional.

"The NAO report into the National Programme for IT involved a lengthy and highly detailed study, which took numerous inputs from a variety of quarters. It is entirely reasonable that senior officials should check with the NAO the status of information and inputs they are receiving given the volume and variety of information collected."

It added, "The UK Computing Research Council's engagement with the NAO is a matter for the NAO."

Warner said he is pleased with the final NAO report. So is Connecting for Health, which has cited the final NAO report as an endorsement of its work on the NPfIT.

But some may be left questioning whether the NAO's final report on the NPfIT was as robustly independent as the audit office's reputation. They may also ask why Connecting for Health seemed so concerned about a third party review of the NPfIT.

The NAO is to publish a new report on the NPfIT.

Key questions experts asked about NHS's £12.4bn IT programme

In 2005 the UK Computing Research Committee, which is made up of leading computer experts, submitted a paper to the National Audit Office, raising questions about the NHS's £12.4bn National Programme for IT. The paper triggered a letter from Richard Granger, IT head of the NPfIT, to the NAO.

These are some of the questions the committee suggested the NAO ask in its review of the NPfIT. Even today, there are still no clear answers to many of the issues raised by the committee.

  • What are the top-level objectives that the new systems are designed to achieve and which, if not all met, would mean that the project had failed?
  • Are there alternative solutions to the identified problems that would have a better ratio of benefits to costs than the proposed IT solutions?
  • How widely will the proposed systems have to be deployed and used before they can achieve their planned benefits at a level that justifies the costs?
  • How have you ensured that the stakeholders really understand how the new systems will affect their working processes?
  • How long could access to data be permitted to take before the delays became unacceptable in the most time-critical of the required functions?
  • What estimates have been made of the cost and impact on NHS services of designing new workflows and processes, and training staff to use the new systems?

The paper was submitted to the NAO on behalf of the committee by Martyn Thomas, a visiting professor in software engineering at Oxford University, who is among 23 academics who are calling for an independent technical audit of the NPfIT.

The committee comprises an expert panel of the British Computer Society, the Institution of Electrical Engineers and the Council of Professors and Heads of Computing. It was formed in November 2000 as a policy committee for computing research in the UK. Its members are leading computing researchers from UK academia and industry.

British Computer Society expresses concerns about the NHS IT programme

The final report of the National Audit Office on the NHS’s National Programme for IT (NPfIT) includes
few specific criticisms of the scheme – despite detailed concerns expressed to auditors by third-party organisations.

Glyn Hayes, chair of the British Computer Society’s Health Informatics Forum, for example, has disclosed that the BCS contacted the NAO about what it regarded as the “monolithic central National Spine”, which is being designed to hold medical records on 50 million patients in England.

Hayes wrote to Martyn Thomas, one of 23 academics who have written an open letter to the House of Commons Health Committee, calling for an independent technical audit of the NPfIT.

The BCS sent Computer Weekly a copy of Hayes’ letter to Thomas. It stated, “We at BCS Health Informatics Forum have always been in favour of such a technical review.

“There are aspects of the project, in particular the monolithic central National Spine, which worry us greatly.

“This was one of the points we made strongly to the NAO.

“Obviously such a centralised system could be made to work technically, although they are having major problems doing so because of the scale and scope of the NHS.

“Our main concern being that a centralised system will not work in the complex organisational structure of the NHS. A distributed architecture would have been much more flexible.”

This specific concern of the BCS was not included in the NAO’s report on the NPfIT.

The Health Committee is expected to hold an inquiry into the programme.

 

Vote for your IT greats

Who have been the most influential people in IT in the past 40 years? The greatest organisations? The best hardware and software technologies? As part of Computer Weekly’s 40th anniversary celebrations, we are asking our readers who and what has really made a difference?

Vote now at: www.computerweekly.com/ITgreats


Read more on IT risk management

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.

-ADS BY GOOGLE

SearchCIO

SearchSecurity

SearchNetworking

SearchDataCenter

  • How do I size a UPS unit?

    Your data center UPS sizing needs are dependent on a variety of factors. Develop configurations and determine the estimated UPS ...

  • How to enhance FTP server security

    If you still use FTP servers in your organization, use IP address whitelists, login restrictions and data encryption -- and just ...

  • 3 ways to approach cloud bursting

    With different cloud bursting techniques and tools from Amazon, Zerto, VMware and Oracle, admins can bolster cloud connections ...

SearchDataManagement

Close