NAO: The challenge of objectivity

Health minister Lord Warner is relaxed when answering the toughest questions on the NHS’s National Programme for IT [NPfIT].

Health minister Lord Warner is relaxed when answering the toughest questions on the NHS’s National Programme for IT [NPfIT].

Being a minister he can answer in any way he chooses, whether or not his response has much bearing on the question.

Two weeks ago week at a press conference in Whitehall, a respected BBC journalist asked Lord Warner whether a report on the national programme by the National Audit Office, had been a “bit of a whitewash”.

The journalist had observed that the report was “less barbed than expected but was also a year late". He questioned whether Connecting for Health, an agency of the Department of Health which runs the NPfIT had played a part in the delay.

His question arose because of a Parliamentary convention which ensures that reports of the National Audit Office are published only after the department investigated has signed off the document as factually accurate.

If officials raise many queries on draft reports, they can delay the final publication date. In the 1990s, a Whitehall official conceded at a hearing of the Public Accounts Committee that the Department of Health had contributed to the delayed publication of a report by the National Audit Office report on an IT-related project Read Codes.

At last week’s press conference the BBC journalist said, “We were advised at the National Audit Office that the reason [for the delay in publishing the report on the NPfIT] was that the draft was with Connecting for Health.”

He understood that there had been an “almighty tussle” over the contents of the report, “with the obvious implication that what has emerged is a bit of whitewash”. He added that Connecting for Health appeared to have emerged victorious from the tussle.

Lord Warner did not seek to contradict the journalist’s claims of arguments over the contents of the report, or that it was a bit of a whitewash, and he deftly answered the question by promoting the NPfIT.

He said, “It seems important in these projects – and the National Audit Office recognises this - that there is both factual accuracy and an uptodateness about the report in terms of the information that is in it. This is a pretty fast changing project. All sorts of things are now beginning to happen quite rapidly.”

He added that his department had a duty to ensure that the National Audit Office understood the “complexity of some of the figures and numbers in this area” and any delay was not about ensuring that Connecting for Health emerged triumphant.

Some in the NHS disagree with the BBC journalist: they regard the report as a comprehensive whitewash, not a bit of one. But the apparent reluctance of the National Audit to criticise the programme – in the face of deep concerns which are expressed by directors of a number of boards of NHS trusts – is in some ways understandable.

It is rare for the National Audit Office to investigate a live project. Its auditors usually pore over the archived papers of projects that ended, for good or ill, some time before.

By the time the audit office publishes a report on a dead project, officials in Whitehall and ministers will be much less sensitive to criticism than if the project were continuing. Probably, the officials and ministers involved will have moved to other posts.  

In the case of the report on the NHS’s national programme, the Prime Minister Tony Blair, who set the NPfIT in motion, is still in office. He spoke with enthusiasm earlier this month (June) on the NPfIT and the positive messages in the report of the audit office.

So a critical report is likely to have angered the Prime Minister who chaired a meeting at Downing Street in February 2002 when the NPfIT was given a provisional go-ahead.

Only five months after this meeting, in July 2002, the NPfIT had successfully passed its initial “gateway review” by the Office of Government Commerce. Gateway reviews are independent strategic assessment of IT-related programmes at various stages in their life. 

Being a programme that is costing £12.4bn or more – the largest IT programme undertaken by the UK government – political sensitivities to criticism are particularly high. 

And while a large project is continuing, auditors do not want to be seen to be critical in a way which could affect morale. The National Audit Office falls in firmly behind Connecting for Health in criticising negative media of the NPfIT.

“There is a risk that negative reporting about the programme and its progress will tend to increase the level of NHS staff scepticism about the programme,” said last week’s report of the National Audit Office. 

So nobody should be surprised if the National Audit Office wants to avoid being seen to add to negative perceptions of the programme. However most major failures of  IT-related projects commissioned by various governments have at least one thing in common: a hypersensitivity to criticism and a suppression of dissent. 

An independent report by consultants Arthur D Little on a long-delayed project to build air traffic control systems for use at Swanwick in Hampshire, for example, criticised the culture of secrecy and management’s unwillingness to face up to bad news.

A further reason for a reluctance to criticise might have been the difficulty of obtaining concrete facts.

Rather than dip hands into the scalding hot stew of commentary, reports and papers of the many critics of the programme, it is safer to go to a computer and read files from an unassailable source: Connecting for Health. To obtain supplementary information, the National Audit Office has sought the help of, for example, those working on the programme in regions and clusters.

According to the “Methodology” section of the report, the National Audit Office had little direct contact with the hundreds of NHS trusts and GP practices.

The report said the audit office visited two of the early roll-out sites to “identify examples of what the programme is delivering in practice.” This is one of the only references in the report to trusts having contributed directly to the report.  

Yet it will be the widespread use of technology in trusts and GP practices, and the changing of working practices to suit the new systems, that will largely determine the success of the NPfIT.

The National Audit Office had received a plethora of submissions, including papers from specialists, MPs, and organisations such as the British Computer Society and the Worshipful Company of Information Technologists.

Many of the submissions contained some cogent criticisms of aspects of the programme. But the report of the National Audit Office appears to attach little significance to the concerns.

In understanding what is and what is not important the National Audit Office is likely to have been helped by having as an observer one of its senior executives on the programme board of Connecting for Health for two years.

This has its pros and cons. The placement gives auditors an invaluable insight into the programme’s vicissitudes. It also helps the audit office to give Connecting for Heath’s executives an insight into the lessons from other government IT-related projects.

But it leaves the National Audit Office vulnerable to a perception of a potential conflict of interest. Could the National Audit Office criticise a programme which has been advised by one its senior executives, even if he did not take part in decision making?

In the end, the audit office has published a report which contains less criticism of the programme than a speech or media interview by Richard Granger, Chief Executive of Connecting for Health.  Where the report does criticise, its comments are mild or counterposed by a positive statement. In contrast, the praise is effusive. 

It appears, then, that the National Audit Office has faced one of its toughest ever challenges in producing a report on the NPfIT. If its draft report had been critical, this is likely to have incited the anger of Connecting for Health, with the possible result that the document might have gone unsigned by health officials.

Had the report been published without any sign-off from the department, this would have been a rare departure from convention, and a potential cause of bitter exchanges over facts when auditors and Whitehall executives came together at a subsequent hearing of the Public Accounts Committee over the progress of the NPfIT.

The much safer alternative for the National Audit Office has been to publish a largely uncritical report. It is replete with praise for almost every aspect of the programme, and its findings have been welcomed by the government as fully supportive of an important and costly initiative, the largest IT programme undertaken in the UK public sector.

The benignity of the report avoids any possible confrontation with Connecting for Health over whether the facts in the report are up to date and accurate.

The only disadvantage is that a BBC reporter asks the Health minister an awkward question about a possible whitewash and one or two MPs become suspicious that the National Audit Office has gone native or become politicised in this instance.

In their formal responses, Connecting for Health and Lord Warner took issue with not a single sentence in the report of the National Audit Office. Lord Warner said, “This is a very positive report which confirms that the programme to modernise the NHS’ computer systems is much needed, well managed, based on excellent contracts, delivering major savings, is on budget [and] has made substantial progress.”

But if the National Audit Office has writes an unbalanced report, which reflects the strengths of a programme and few of the weaknesses, is it doing its job well?

Read more on IT risk management