Data tyranny to blame for Leeds

NHS medical director Sir Bruce Keogh may have breached his own data safeguards when he used preliminary data to justify the suspension of children’s heart surgery at Leeds NHS Hospitals Trust on 28 March.

Keogh closed the unit after requesting provisional data to be leaked from the national heart data centre, the head of data at the centre has told Computer Weekly.

It has also emerged that as Keogh himself founded the very unit that produced the leaked data, he should have known it was incorrect. He should have known it could not be used to justify the Leeds closure. He may have further misled the public when he later blamed his mistake on Leeds, for what he said was its release of inaccurate data.

Dr David Cunningham, senior strategist at the National Institute for Cardiovascular Outcomes Research (Nicor – the data unit Keogh founded), said it had been obvious to experts charged with verifying the preliminary data that there was something wrong with it – something that stopped any of them doing anything with it.

Those experts – called the steering group – looked at the data when it was released on 27 March.

“They looked at it and said, ‘That’s too low, can we check it?’ They saw the numbers didn’t look right for Leeds that year,” said Dr Cunningham.

The thing that didn’t look right about the Leeds data was not what Keogh claimed it was when he saw it that same day.

It was obvious to the Nicor experts that the Leeds data was incomplete. This clearly meant it could not be used. It described a fall by more than a third in the number of operations done at the hospital in the last year. There was clearly something missing.

“That’s the purpose of sharing it around the committee. It’s for people to look at and say, ‘That doesn’t look right’,” said Dr Cunningham.

“It was obvious a query should be raised about it. So I was asked to get people to check it. And that’s what we did. It took about 24 hours,” he said.

This procedure was an established routine for health data at the unit Keogh founded, and is an annual event.

On 27 March this year however, Keogh requested a copy of the preliminary data, said Cunningham.

Sir Roger Boyle, who resigned as head of Keogh’s Nicor unit this week, sent Keogh the preliminary data. Both he and Keogh have confirmed that Keogh knew it was preliminary.

What was not clear until now is that Keogh, as Nicor founder, would have been familiar with the data and the validation procedures. He would like the other Nicor experts have known the data looked unusual enough to be verified before taking any action. He would have known Nicor’s routine procedures for verifying unusual-looking data. He would have known to follow those procedures before taking the action he did.

“That’s quite a serious allegation,” said a spokeswoman for NHS England, where Keogh is now medical director. She was not able to answer the allegation. But she said there had been other reasons for Keogh’s decision to close the unit. Keogh had nevertheless told BBC Radio 4’s Today Programme the data had been his main reason for ordering the Leeds unit’s suspension.

Keogh got hold of the preliminary data late on the afternoon of 27 March, shortly after it had been sent to the Nicor steering group usually charged with looking over it. By the following morning he had already ordered an emergency suspension of the Leeds unit. He went to Leeds to do this in person.

In the time it took Keogh to do this, Nicor’s steering committee of eminent clinicians had verified that their immediate suspicions about the data were true. It was inaccurate.

Keogh failed to see what Dr Cunningham said had been obvious to the other experts. He took the erroneous data at face value.

The error had made it seem as though the data showed around a 100 per cent increase in deaths at Leeds in the space of a year. It seemed that the rate of mortality at Leeds had gone almost overnight to 200 per cent the national average. Leeds had simultaneously seen a highly unusual fall in heart operations, according to the preliminary data.

It should have been clear even to a layman that something was more likely wrong with the data than with Leeds. The Nicor data experts new within 24 hours it was a problem with the data. They knew this because they checked it. Keogh claimed the latter could not be ruled out and closed the unit for nine days while he reviewed the situation.

When on 9 April Keogh reopened the unit, he blamed Leeds for providing him with inaccurate data. Jeremy Hunt, the health secretary, supported Keogh on this. Yet the missing data was found quickly after Nicor told Leeds it was missing. The data error was picked up in a routine procedure by the Nicor unit Keogh established. It was all above board. Leeds could not even be chastised for having produced inaccurate data, as Keogh and Hunt had attempted.

Dr Cunningham told Computer Weekly the steering group’s receipt of the preliminary data on 27 March was the “first pass” in a routine procedure for verifying data that has been collected annually since 2000.

“We don’t circulate preliminary analysis that might have flaws in them. That would be a recipe for drawing the wrong conclusions,” he said.

Keogh, first by requesting unverified data, then by ignoring the data’s obvious faults, then by breaching usual procedure in not seeking to clarify those faults, then apparently wilfully misinterpreting that data, and then using the weight of all these errors to tell Leeds to close its heart unit, may have committed a serious breach of protocols he had helped establish.

Keogh was unable to clarify just how familiar he was with Nicor’s precautionary procedures.

His office tried to claim he was not a data expert at all. Yet keogh has been one of the foremost health data experts since he founded the National Adult Cardiac Surgical Database in 1995. This database was the precursor to the Nicor unit. He should know how to read its data, and how not to misrepresent it.