Highlights of National Audit Office NPfIT report

These are excerpts from today’s report [16 May 2008] by the National Audit Office on the NHS’s National Programme for I.T

Much has been published by Connecting for Health on the achievements so far of the NPfIT. The excerpts here highlight some of the important lessons to be learned from the challenges of implementing the NPfIT.

Huge effort and commitment by NHS staff

(par 36 , page 13) During our visits we saw that NHS staff are demonstrating huge effort and commitment to make deployments go as smoothly as possible, and we saw clear evidence of Trusts learning from the experience of others.

Care Records Service four years later than planned

(page 9) Current indications are that it is likely to take some four years more than planned – until 2014-15 – before every NHS Trust has fully deployed the care records systems. Until Lorenzo is available and has started to be deployed, there remains a particular uncertainty over timing in the North, Midlands and East. Good progress is being made with other elements of the Programme.

Is the NPfIT going to become even more complex?

(par 1.14, page 20) Government policy is to use the most appropriate or cost-effective health care provider, whether a part of the NHS or an independent organisation such as a Treatment Centre. Bringing these organisations into the Programme will add further complexity …

Security and smartcard sharing – new risks

(Par 3.27, page 37) Since April 2006, five security incidents have been reported to NHS Connecting for Health, although two of the five were subsequently found to be failures of process rather than system incidents. The remaining three cases arose when authorised users of the systems found they could view more patient data than they should have been able to view…

(Par 3.79, page 45) The Trusts we visited confirmed that Smartcards were being used as intended. Some Trusts highlighted, however, that system performance had important implications for the effectiveness of the Smartcard arrangements. If it took staff a long time to log into the system using their Smartcard, they might be increasingly minded to leave their card in the ‘reader’ when they went for a break or to share cards rather than logging in with their own card, in contravention of the rules.

(Par 3.78, page 45) Two Trusts reported concerns that they were not currently able to obtain reports locally that showed the role of each Smartcard holder and an audit trail of any changes, although this report can be produced centrally.

(42f, page 14) Early experience with the Summary Care Record indicates that patients have a high level of confidence that their personal data will be secure, but security lapses could easily undermine that confidence and reduce the benefits of the Programme.

(conclusion, page 12) Greater sharing of patient records brings new risks. Ultimately security depends on the actions of individual NHS staff …

When something goes wrong, much scope for “not my fault” syndrome

(par 2.44, page 30) One of the challenges of assessing the performance of the Programme’s systems is the complexity of pinpointing where the cause of a particular problem lies. For example, for a GP to make a Choose and Book appointment for a patient depends on the Choose and Book software itself, the N3 network, several components of the Spine, and the many components of local GP and hospital systems all working effectively. A failure in any one of the applications can result in the Choose and Book transaction not being completed. The interdependence of the systems and the lack of visibility at local level of the process from one end to the other can also mean that the data may not bear out the perceptions of NHS staff about the performance of particular systems.

And things did go wrong – 807 severity 1 and 2 incidents in 6 months

(par 2.48, page 30) In the six months from July to December 2007, there was a total of 807 severity 1 and severity 2 incidents, which were attributed to the Programme’s suppliers [71 of which were not fixed within designated time limits].

(par 2.47, page 30) Examples of severity 1 incidents, the most serious, include: a hospital system (such as a care records system or a Picture Archiving and Communications System) being completely unavailable at one or more hospitals; a loss of power at a Local Service Provider data centre, causing the failure of multiple services; and a high risk clinical safety or information governance issue, such as a patient record being displayed with the demographic details of another patient. Severity 2 incidents are less serious or extensive service failures, such as a number of users at a hospital are unable to access a hospital system, or a partial loss of functionality at a hospital which has a significant impact on hospital processes and for which there is no known workaround.

A long time to fix problems – sometimes not before a new software release

(par 35, page 46) All the Trusts we visited had experienced some technical problems with the new care records systems, although the nature of the problems varied. All Trusts had procedures in place for staff to raise issues and for reporting them to the Local Service Provider. Trusts reported, however, that resolving issues often took a long time and in some cases could be done only through a later release of the software. A particular issue for Trust staff using the systems was a lack of visibility of the resolution process, for example knowing what the timeline for resolving a particular issue was and what progress had been made.

Suppliers finding it tough

(par 3.59, page 42) One Local Service Provider likened the deployment of a new care records system in a medium to large Trust to the replacement by a major retailer of its whole supply chain system, involving hundreds of stores and every supplier and all the processes in-between.

(par 34, page 12) The three Local Service Providers [CSC, Fujitsu and BT] told us that the scale and complexity of the Programme made it extremely challenging. They described how it can be difficult to plan and deploy resources where progress relies on many decisions necessarily made at local level, and how they cannot make progress simply by ‘working to the contract’ but need to be highly flexible to meet NHS requirements [comment: the centrally-driven Wessex Regional Health Authority Regional Information Systems Plan failed in 1992 after local trusts decided not to support it]

(par 2.29, page 26) Under the terms of the contracts, suppliers are paid only when services are proven to have been delivered and working for 45 days. For the new care records systems, NHS Connecting for Health does not pay Local Service Providers until it receives agreement from the Trust concerned. In some instances, suppliers have not been paid for over 12 months after the deployment of systems.

(par 2.30, page 7) At 31 March 2008, Fujitsu in the South had the highest average and the most deployments for which no payment had yet been made. In respect of the care records systems, however, the deployments in the South and also in London were early releases of the final system, whereas in the North, Midlands and East to date only interim systems have been deployed as Lorenzo is not yet available. Fujitsu told us that, in its view, Trusts on occasion held back from agreeing that payment should be made, even where systems were working, until all the non-contractual changes they were seeking had been implemented.

(par 3.49, page 41) …. the care records systems being deployed allow a degree of flexibility, and an individual Trust may make a hundred or more change requests before it is prepared to sign off the system as meeting its requirements.

(par 3.54, page 41) All the Local Service Providers have found it difficult to plan and deploy their resources on a Programme where progress relies on many decisions necessarily made at local level. In this respect, the Programme differs from other major programmes that they have been involved in, in both the public and private sectors….

Part of CSC’s £8.5m fine for data centre crash put off until end of contract

(par 2.50, page 31) The largest performance failure to date concerned CSC’s contract as the Local Service Provider for the North West and West Midlands. A 45-minute power outage at CSC’s data centre in Maidstone on 30 July 2006 was followed by problems restarting systems. Data held in the data centre could not be accessed and services could not immediately be provided by the back-up systems that had been put in place. No data was lost but 80 NHS Trusts were affected and had to operate paper systems as a contingency. Services were restored on 2 August 2006, with resolution of all the issues completed on 8 August 2006. CSC incurred a performance penalty of £8.5 million for this incident, of which £1.2 million was deducted at the time, with the remainder to be used to buy additional services free of charge or deducted at the end of the contract.

Whitehall may be unable to force trusts and others to buy NPfIT systems

(par 1.11, page 19) In most programmes to introduce IT systems … the governance structures allow the key aspects of the programme to be applied compulsorily. This is not the case for the National Programme for IT because the NHS is highly devolved. The main organisations, the Trusts, which are taking the new systems, are self-managed with their own governance arrangements. Their Boards and Chief Executives make the final decisions about deploying the new systems in their Trust.

(Par 1.14, page 20) …some [NHS trusts and other health organisations such as treatment centres] may have business reasons for using different core systems from those offered under the Programme.

Foundation trusts could go their way – outside the NPfIT

(Par 1.12, page 19) Furthermore, 88 Trusts are now Foundation Trusts, which are autonomous though still part of the NHS. They are independent, not-for-profit public benefit corporations … and the Secretary of State for Health has no powers of direction over them. The Trusts have members drawn from patients, the public and staff and are governed by a Board comprising people elected from and by the membership. The Government is committed to offering all Trusts the opportunity to apply for Foundation Trust status.

Suppliers receive compensation when trusts don’t buy NPfIT systems

(par 1.14, page 20) The Programme’s contracts were based on all NHS Trusts taking the systems at some point. Foundation Trusts cannot now be forced to take the systems and, should any elect not to do so, there will be financial implications for both the Trusts concerned and the Programme (paragraph 2.32). The NHS Chief Executive has, however, directed the Strategic Health Authorities to ensure that other NHS organisations in their areas meet the expectations of the contracts.

(par 18, page 9) In the event that the Local Service Providers do not receive the expected revenue for reasons solely due to the Department (for example, where a Trust elects not to deploy the system), the Department has to make a payment to the supplier. At 31 March 2008, payments totalling £36.1 million had been paid under these arrangements.

New systems may provide less initially than old technology

(par 2.18, page 23) NHS Connecting for Health has committed that Trusts will not be expected to take the Programme’s systems until they are at least as good as the systems they currently have. In practice, however, if existing systems are old and/or no longer being supported by the supplier concerned, Trusts may have little choice but to take the systems offered under the Programme and this can result in a loss of functionality, at least in the short term…

First early adopter – Bolton – stretched despite extra help

(par 2.4, page 22) During our visit to Bolton, we found that deployment of the Summary Care Record had generally gone smoothly, although progress was slower than expected. The roll-out had meant substantial extra work for the Primary Care Trust and, although NHS Connecting for Health had provided considerable support including a dedicated project manager, the Trust’s resources were stretched…

Some extra costs and delays for trusts after deployments

(par 2.40, page 29) Some Trusts we visited had identified savings arising from the deployment of the new systems, but there was also evidence of operational performance declining immediately following a deployment. Some staff had not found the new care records systems intuitive to use and key processes such as booking a new patient into an outpatient clinic were taking much longer than they had previously. This had prompted some Trusts to take on additional staff to input or process data…

On CSC, “Lorenzo” and why it’s late [originally promised in 2004]

(Par 2.10, page 22) The development of Lorenzo has taken much longer than originally planned, with the delays attributed in part to an underestimation by all parties of the scale and complexity involved in building a new system from scratch.

(par 2.11, page 22) In the light of concerns about progress in developing Lorenzo, in summer 2007 NHS Connecting for Health and CSC jointly commissioned two reviews of the delivery arrangements. The first review, by experts in IT development, identified, among other things, a lack of clarity around responsibilities and shortcomings in programme management and end-to-end delivery arrangements within CSC and iSOFT. It concluded that further delays could be expected. The second review, by a large-scale systems integrator, reached similar conclusions, in particular noting that the then current plan for development and deployment was behind schedule and in any event not feasible, and recommended that the phased strategy that NHS Connecting for Health had suggested should be adopted.

(par 2.12 -2.13, page 23) … the planned two releases of Lorenzo have been broken down into four smaller releases. Nevertheless there remains considerable uncertainty over the delivery schedule for Lorenzo.

Par 2.14, page 23) While the most deployments have been made by CSC in the North, Midlands and East, these are of the iPM system, an existing system upgraded to meet the requirements of the Programme, which offers limited clinical functionality. iPM is an interim solution …

Contracts with local service providers unrealistic?

Par 2.15, page 23) The timetables the Local Service Providers originally agreed with NHS Connecting for Health proved to be unachievable given the scale of the challenge involved in developing and deploying the care records systems in the NHS

Some trusts prefer the back of the NPfIT queue

(par 21.17, page 23) Many of the care records systems deployed to date offer limited clinical functionality … In some cases, Trusts have decided to wait for the later releases and not take the limited clinical functionality available in the first release.

Further uncertainties

(conclusion, page 12) Until the process of contract resetting [revision of parts of a contract with Fujitsu] is complete, there remains a degree of uncertainty in relation to the South.

Unrealistic expectations

(conclusion, P 13) The original unachievable timescales for the Care Records Service as a whole have been mirrored in the deployment of the care records systems at local level, and raised unrealistic expectations at times.

(overall conclusions, page 13) All elements of the Programme are advancing and some are complete, though delivering a nationally specified Programme into the highly devolved NHS continues to be an enormous challenge. For the Care Records Service, the original timescales proved to be unachievable, raised unrealistic expectations and put confidence in the Programme at risk.

[par 7, page 8) To support the creation of Detailed Care Records, the Local Service Providers (BT in London, Fujitsu in the South and CSC in the North, Midlands and East) are implementing electronic care records systems in a series of releases. The scale of the challenge in developing and deploying these systems in the NHS has proved far greater than expected, and the timescales the Local Service Providers originally agreed with NHS Connecting for Health proved unachievable.

(Par 3.61, page 43) The planned timetable for deployment had not been achieved in the majority of Trusts we visited, in some cases repeatedly, for example because systems had not passed successfully through testing. In retrospect, most Trusts that had experienced delays considered they, and the Local Service Provider concerned, had substantially underestimated the time and work required to deploy the new system.

(Par 42b, page 14) 8, page 8) The delays in developing Lorenzo [the main NPfIT software for trusts in the north of England, east and West Midlands] make it even more important to get the product right and win the confidence of NHS staff. Current plans are to have the first release available for deployment at three early adopter Trusts in summer 2008, with full roll-out planned from autumn 2008. Given the experience of deploying other care records systems within the Programme, however, this timeframe may prove over-ambitious. [Angela Hands, an author of the NAO report, said the experience of NHS staff at trusts which have installed Cerner systems is that it takes a year or more between drawing up plans and going live. Therefore, she said, it may be optimistic for the first trusts to go live with Lorenzo this summer and then for the rollout to happen in the Autumn. The lessons from the first Lorenzo go-lives should be learned before further implementations are planned, even if this extends the completion of the NPfIT beyond the current projection of 2014-15.]

Honest communications – the need to consider drawbacks as well as benefits

(recommendations for trusts, page 15) Trusts should establish in detail the advantages and disadvantages of the system being deployed compared with the one being replaced, and clearly communicate them to staff in order to manage expectations.

(Par 3.66, page 43) In all the Trusts we visited, the new care records systems brought advantages and disadvantages compared with those they were replacing, and they were inevitably taking some time to work as intended following deployment. In some Trusts, the old systems had been developed over many years, often with the direct involvement of Trust staff. Though these systems were judged unable to support the aims of the Programme (for example, in terms of sharing care records with other parts of the NHS) and were unsustainable in the long term, they did meet the specific needs of the Trust at that point in time. It was therefore common to find that some staff felt a sense of loss in moving to a new system that could only be customised to some extent and over a period of time. This is a frequent response in change programmes, but can be managed by recognising the new system’s disadvantages compared with the old system, and acknowledging them to staff during training and in communications about the Programme.

The NPfIT “still appears feasible

(overall conclusions, Page 13) The original vision for the Programme nevertheless remains intact and still appears feasible. The major outstanding challenge is to finish developing and deploying the care records systems that will help NHS Trusts to achieve the Programme’s intended benefits of improved services and better patient care.

Costs uncertain – and an increase of £678m

(Par 14, page 9) Since the start of the Programme, there has been an increase of £678m (11 per cent) in the value of the core contracts, due mainly to the purchase of increased functionality … The remaining increases on the core contracts have resulted from supplier and sub-contractor changes…”

(conclusion, page 9) It remains difficult to produce a reliable estimate of local costs.

(par 2.21, page 24) The cost estimates are un-indexed and made at 2004-05 prices; the cash outturn will be higher due to the impact of price inflation in years subsequent to 2004-05.

(pars 2.27-2.28, page 26) Expenditure on the Programme totalled £3.55bn in resource terms at 31 March 2008 [of which £1.9bn was on the LSP contracts and the rest on central and some local costs]

Some dissatisfaction with installed systems

(par 25, page 10) All the Trusts we visited had experienced some technical problems with the new care record systems, and there had been some dissatisfaction, especially in the period following a deployment, as is often the case with IT programmes. Many staff had come to prefer the new system to the one it had replaced, though some continued to be dissatisfied, for example where issues they had raised had not yet been dealt with.

(Par 3.84, page 46) Many staff found that the versions of the systems that had been deployed were less intuitive than they would have liked and it took longer to record initial patient information than it had done previously, in part because more information needed to be captured for reports to be generated automatically. In many cases, practical problems resulted in staff using the systems in different ways within the same Trust. Such lack of consistency can, however, affect the accuracy of reporting if information is not entered correctly…

The NPfIT Local Ownership Programme – accountability not straightforward

(Par 27, page 11) The Local Ownership Programme has been widely welcomed by people working in the NHS and other stakeholders, although its impact has in the main yet to be felt. In the highly devolved NHS, the practical reality for the Senior Responsible Owner for the Programme and for the Strategic Health Authorities’ accountability in their areas is not straightforward. So, for example, decisions about when a new care records system should be deployed lie with Trust Boards and their Chief Executives, rather than with the Strategic Health Authorities.

NPfIT spin

(Par 3.4, page 32) On a Programme of this size and complexity, communications are both vitally important and difficult, and the people we interviewed confirmed the importance of getting communications right to help maintain confidence in the Programme. At national level, most communications come from NHS Connecting for Health specifically rather than the wider Department. NHS Connecting for Health has its own website which makes available a considerable volume of information about the Programme’s achievements, together with information on service availability, guidance for the NHS and communications for patients. During the course of our work, however, concerns were raised about a need for greater openness and realism in presenting what remained to be done as well as what had been achieved

(Par 28, page 11) Large volumes of data are available to help manage the Programme, though communications have tended, to date, to focus on achievements rather than what remains to be done. Our difficulty, in producing this report, in collating the Programme’s current position to a reasonable degree of precision, reinforced our impression that reporting and communications about the Programme could be improved, particularly in relation to the deployments by the Local Service Providers

Par 3.29, page 37 NHS Connecting for Health modified the content of the third survey [of NHS staff and clinicians] …Among the changes was that NHS staff were no longer asked how favourable they were towards the Programme so views on this question, which we covered in our previous report, cannot be tracked over time.

Some lessons so far – and workarounds

(par 3.71, page 44) Most Trusts considered, in retrospect, that they should have done more work to map processes, which would have identified more potential pitfalls and reduced problems or brought earlier benefits after the deployment. In some cases staff were devising workarounds to make the system work with their processes in the way previous customised systems had, and almost all Trusts needed to do additional work subsequently to make sure that staff were using the new system as intended.

(par 3.86, page 46) … Some staff were working round issues themselves rather than reporting them, and … people highlighted problems that were not previously known to the Trust management.

Support for the NPfIT – not all staff convinced of the benefits

(conclusion, page 12) The arrangements for engaging with clinicians and NHS staff, and involving them in the development of the systems, have been strengthened …There is, however, still progress to be made before all staff are convinced of the benefits of the Programme

(Par 39, page 13) Although increased functionality is planned for later releases [of the Care Records Service], the limited clinical functionality provided to date had made engagement [with clinicians] more difficult.

(Par 2.18, page 23) Two Trusts we visited in the South described how the first release of Millennium had less functionality than their previous systems and how this had made it particularly difficult to engage with clinical staff…

(Par 3.65, page 43) Trusts found, however, that it was difficult to engage their clinicians in a meaningful way when the new care records systems as yet offered little clinical functionality and when there was no realistic training environment for staff to use.

(par 3.61, page 43) The slippage had sometimes had serious implications for staff engagement and training, but it had been necessary to delay the ‘go live’ date to achieve a smooth transition.

(par 1.14, page 20) The enthusiasm and effort needed to derive the Programme’s benefits will have to be driven locally and cannot be mandated from the centre.

(par 3.83, page 45) … One Trust had replaced an elderly but fully integrated administration and clinical system with a new care records system and several non-integrated clinical systems; this change had a very negative impact on the Trust’s ability to engage clinical staff

(Par 3.34, page 38) Lack of engagement with users of the systems was seen by the external bodies we consulted as one of the main risks to delivery of the Programme. Some bodies considered that engagement had improved in recent years but most felt there was scope for further improvement to increase the likelihood of the Programme’s success…

Training systems different to live technology

(par 38, page 13) … the training environment provided to Trusts differed from the live system they were deploying.

(Table, page 15) The Department should require Local Service Providers to provide Trusts with a training environment as close as possible to the system being deployed. For example, the training environment should reflect the different structures of a Trust’s outpatient clinics, to reduce the risks associated with staff having to use a system that looks different from the one they trained on.

Criticism by NAO of Fujitsu’s national systems helpdesk

(par 40, page 13) During our visits [to NHS trusts which had installed , feedback was that the performance of the Service Desk was universally poor. NHS Connecting for Health and Fujitsu recognise there have been problems with the operation of the Service Desk and are taking steps to improve performance.

Choose and Book – a key part of the NPfIT – bookings well below expectations but a success at some primary care trusts

(par 41, page 13) While the Choose and Book system is now nearly fully deployed, utilisation has been lower than expected, with 6.7 million bookings, against an original forecast of 39 million, by January 2008. Usage has been rising, and around half of new outpatient appointments are now being booked through Choose and Book, though there is wide variation in utilisation rates between Primary Care Trusts, ranging from over 90 per cent to below 20 per cent.

(par 3.95, page 48) GPs have also raised issues with NHS Connecting for Health about system performance when accessing Choose and Book from their local system through the Spine. Some have found it very slow to access the system and book appointments and consequently impractical during a patient consultation… local IT configuration, for example within the GP surgery itself, could dramatically affect the speed with which bookings could be made.

No transparent means of handling system performance problems

(Table, page 15) The Department should require Local Service Providers to have transparent processes for logging and dealing with system performance issues and for handling change requests, thereby enabling Trust staff to monitor progress.

Need for better checks on whether promised improvements have happened

(Table, page 15) The Department [of Health] should check whether the planned improvements to the service provided by the NHS Connecting for Health Service Desk to NHS staff have been fully and effectively implemented.

BMA warns some GPs may refuse to upload patient records

(par 3.19, page 35) GPs also need to be assured that the confidentiality of their patients’ care records will be protected and the British Medical Association told us that if concerns around the safety and security of patient information were not answered, GPs might refuse to upload their patients’ records.

Has NHS Connecting for Health given up NPfIT surveys of doctors?

(3.28, page 37) To track awareness and understanding of the Programme in the NHS, NHS Connecting for Health has commissioned Ipsos MORI to carry out a series of surveys. We presented results from the first survey, conducted in July 2005, in our first report. A second survey followed six months later in February 2006 and a third in May 2007. At the time of our work, NHS Connecting for Health had not determined its plans for future surveys.

Informal sharing of lessons between trusts – and good practice in London

(par 3.58, page 42) We visited 15 NHS Trusts which had implemented new care records systems under the Programme. Overall our visits demonstrated the commitment of local NHS staff, with many working substantial additional hours during key phases of the deployment process. Earlier deployments have tended to be the most problematic, but we saw clear evidence of Trusts spreading the lessons they had learned, largely through informal networks, which most people felt worked best, although occasionally and increasingly Trusts are sharing resources and expertise. For example, in London, the Local Service Provider, Strategic Health Authority and Trusts are working together to provide some continuity in deployment teams between one deployment and the next.


National Audit Office reports in full

Some NPfIT major issues – did Prime Minister get full briefing in 2007?

Downing Street’s original three-years timeframe for the National Programme for IT

NHS Connecting for Health will stand by Lorenzo

Is government trying to control information on problems after go-lives of National Programme for IT [NPfIT] systems?

Wiki of leading academics who want an independent review of the NPfIT

National Audit Office review of the NPfIT

NPfIT running four years late – Reuters

Four years late for patients’ records – Press Association

In a shock move