Connecting for Health, which runs the NHS’s £12.4bn National Programme for IT, has responded to some our questions. These are its replies.
Computer Weekly question: Severity 1 and 2 incidents – described as “major incidents”, appear to be on the increase. We reported last year on more than 110 incidents in a 4-month period. In a recent 2-month period there were more than 110 severity 1 and 2 incidents, including nationwide issues and failures of PACS systems.
Connecting for Health response:
“The assertions about the increase in the number of severity 1 & 2 incidents are misleading and factually inaccurate. The overall picture is actually showing one of improvement. It would not be accurate therefore to report that the number of incidents is increasing.
“Facts: The number of incidents is not only decreasing in number, but are also decreasing in proportion to the number of users and systems deployed. This means the systems we are delivering are becoming progressively more reliable as we roll out in increasing volume and the platforms become more mature.
“In order to give you a fuller answer, please see the attached information broken down below, which we hope you find helpful.
“Over the past six months the number of deployed systems on the Programme has increased from 10,116 to 13,200, an increase of over 30%. There has been a similar increase in users of some 25%. In the same time frame the number of reported incidents has reduced by 15.3% overall despite the increase in deployments and users. This represents a significant improvement in reliability.
“You should also note that our candour and focus on resolution of Service Management issues which has led us in the past to use terminology such as “major incident” should not be confused with similar language used by other organisations which may have a different meaning.
“For example we would consider a severity one incident to be the loss of a system. An example would be one GP Practice losing system use for five minutes, i.e. the whole of a deployment. This is clearly very different from the significant incident we had in July 2006 when CSC suffered significant data centre outage.
“Inaccurate Reporting of Incidents
“Inaccurate reporting by NHS Trusts of severity one and severity two incidents inflates the initial number of incidents which are recorded and progressed by the National Service Desk. For example approximately 15% of both severity one and severity two incidents which were reported to the National Service Desk over the past six months were in fact identified as being caused by local infrastructure problems. This is hardly surprising given the heterogeneous and complex nature of the infrastructure in many of our Hospitals and GP Practices.
“Similarly 5% of severity 1 and 10% of severity 2 incidents were downgraded upon investigation, i.e. their impact had been incorrectly assessed at the time they were originally reported.
“NHS Connecting for Health appreciates that these factors along with the high level of usage of our systems and their importance in the NHS creates an environment in which we will continue to prosecute a robust Service Management framework which defaults to over reporting of problems in order that they can be properly aired. We believe that this is in line with industry best practice and that it would be unfortunate if inaccurate reporting were to begin to lead to a culture of under reporting of incidents with a commensurate degradation in the quality of root cause analysis.
“You will be aware that our Picture Archiving Systems have now been rolled out to most Trusts across the South of England. End to end service availability is a function not only of the performance of the systems delivered by Fujitsu but also local infrastructure and performance of electro mechanical devices such as CR Readers. Fujitsu is currently undertaking detailed analysis of the run time performance of their equipment and the end to end performance of PACS services. Once this analysis is available Fujitsu are happy to discuss in more detail at one of the live PACS sites.
“Significantly Better than the Industry Average
“It should be noted that not only do our systems perform well within normal tolerance levels according to Gartner with significant incidents being 1% for NHS CfH’ systems compared to an industry average of nearly 5% but also that our level two incidents are running at less than 4% compared to an industry average of approximately 10%.
“The meantime between failure on many of CFH’s systems exceeds contracted levels and in almost all cases the “time to fix” for our systems is within industry standards and contracted timeframes.
“98.6% of severity one and two incidents were fixed within the target timescales and in January 2007, the last month for which full data is available the average amount of time lost due to systems outage (on the presumption that a user was using a system for 24/7 – clearly an exaggeration of the norm) is only five minutes.
“We draw your attention to the service availability data which we publish on our website. NHS Connecting for Health is unusual in publishing this data and would encourage you to compare our performance with other organisations be they public or private sector.
“You should note the high degree of availability of our core National Systems such as the PDS. This is now receiving approximately 1.4 million queries a day from the order of 50,000 unique users.
“This is the backbone of many patient interactions in the NHS today.
Computer Weekly’s supplementary questions for NHS Connecting for Health
Computer Weekly’s question: “Isn’t there a case for CFH publishing the severity 1 and 2 incidents and the lessons learnt from them so that other trusts could gain from adversity?”
Connecting for Health answer: “Yes, we do this already. Details of all HSSIs [higher severity service incidents] are already published throughout NHS CFH with the intention that these are then cascaded as appropriate to the wider NHS in order to provide the ability to identify trends or to pre-empt similar incidents occurring in other areas.
“In addition, analysis of HSSIs and any underlying problems is undertaken by NHS CFH staff. In conjunction with suppliers, work-around, known error and root cause analysis information is circulated in order to help prevent re-occurrence of incidents or to mitigate potential risks to the live service.
“NHS CFH takes any and all opportunities to share lessons learnt. Every HSSI is thoroughly reviewed after the event and any findings are fed into all affected processes and procedures. NHS CFH also promotes and supports a number of user groups to share best practice, common experience and lessons learnt.
Computer Weekly question: “If these problems occur at a relatively small number of sites, what are the implications when there are many more deployments?”
Connecting for Health’s reply: “The assertion that the incidents occur at a relatively small number of sites is an inaccurate misleading and leading statement.
“NHS CfH of course already delivers systems which are ubiquitous in sectors of the NHS such as QMAS.
“Your attention is drawn to our published and regularly updated data on the availability of QMAS.
“Similarly, despite (often inaccurate) media coverage around the performance of the new NHS network, the network in general performs above contracted levels and has delivered a high degree of resilience on a value for money basis.
Computer Weekly question: “Are you confident the Spine, choose & book and national infrastructure will cope well with increasing volumes of transactions?”
Connecting for Health response: “With regard to the performance of the Spine and national messages, we are attaching recent graphical illustrations from BT, which show not only the cumulative volume of messages passing over this infrastructure but also the availability of this infrastructure.
“We would encourage you to compare the performance of this central infrastructure with other comparable infrastructure in other departments.
“Questions regarding capacity of the live estate to accommodate increasing volumes need to be addressed to our suppliers who are contracted to deliver systems which perform in a consistent manner as volumes increase.
“You are of course right to raise the question regarding the risk of critical infrastructure failing as volumes increase. This is something which we are aware of and for which there can be no absolute guarantees. Active risk management and soak testing does of course reduce the probability of such events occurring. However, delivering systems where cost is a consideration in their purchasing arrangements means there is always a possibility, however small, of system failure
Computer Weekly question: “Should Connecting for Health rely on Local Service Providers rather than trusts to report major incidents, as we understand it may sometimes prove difficult for some trusts to convince LSPs of the potential seriousness of the incidents?”
Connecting for Health response: “We do not recognise your statement that Trusts for LSPs have difficulties reporting incidents. We run a culture of openness around system issues and encourage LSPs, NASPs and end-user organisations to report problems as they occur. Mature collaborative working between all parties is necessary to ensure that problems are resolved rapidly.
Computer Weekly question: “It is being said by project management specialists (who do not work exclusively in the public sector or indeed IT) that, given access to the appropriate people and documents, an independent review of the project could be carried out by six people in less than a month, and possibly in 2 weeks or less. Given that the cost would be miniscule in relation to the programme, and that it could identity problems which, if not overcome or different objectives set, could see money being spent without commensurate benefit, why not commission and publish the results of such a review? Is the real objection to a published independent review that it could identify weaknesses with the scheme announced in early 2002 that ministers would not want revealed?”
Connecting for Health’s reply: “The National Audit Office conducted an independent review of the Programme in 2006. Its report was agreed with the Department of Health and is an accurate record of the progress which had been made to June 2006. Significant further progress has been made since that date.
“Former Health Minister, Lord Warner, stated that a further review would neither be constructive nor helpful. The Open Public Meeting which was Chaired by David Nicholson CBE on 26 January, to which a wide range of members of the media and others were invited, provided the opportunity for a further case to be made for a review. No evidence was presented at this meeting
“David Nicholson concluded to all present at the time that “he had not heard any evidence to suggest a review was required.”