Getty Images

Universal Credit: What went wrong, and what we learned

The architect of one of the biggest and most controversial government policies – and IT failures – of the past decade gives his inside view of the project, and how Computer Weekly was a thorn in their side

“No one is going to ask about that – only the geek on Computer Weekly.”

Iain Duncan Smith was clearly angry with me for puncturing his wishful thinking over the reception of yet another announcement of delays to Universal Credit. It was the early autumn of 2014 and Iain was secretary of state at the Department for Work and Pensions (DWP) and I was his minister for welfare reform, responsible for bringing the project in.

We had set out on the project to reform the UK welfare system in 2010. The existing system was a mess – a bundle of often contradictory benefits assembled over the years that categorised and trapped people in the system and in poverty. With Universal Credit (UC), we wanted to sweep this all away and replace it with a system that allowed people flexibility to change their situation in a low-risk way.

The trauma of UC’s introduction proved far beyond anything we had envisaged. In the first five years, we worked our way through no fewer than six project managers and six senior responsible owners, with several collapses and even one death under the enormous pressure.

Throughout this period, Computer Weekly kept a close tally of our travails, fed – we suspected – by hostile informants in the Cabinet Office. It was not surprising that Iain came to dread the next revelation from this source.

IT capabilities

Why was it all so difficult?

At the heart of UC’s troubles lay the capability of the department in IT. It became evident that the commissioning out of IT by government departments a decade and more earlier had effectively removed any direct knowledge of how to build systems, or even monitor that the contractors were building those systems properly.

This did not matter for smaller, discrete pieces of software, where the DWP had developed a good track record. Indeed, Joe Harley, government CIO at the outset of the project, reassured me on the prospects for UC with the example of the successful recent implementation of the new Employment and Support Allowance.  

David Freud

“If departments do not rise to the challenge, there will be plenty more horror stories for Computer Weekly to lay bare”

David Freud, former minister for welfare reform

But this lack of competence meant that the department did not recognise the scale of the system we wanted to build. We were still calling it “an IT development of moderate scale” in the whitepaper of 2010 and beyond.

There were two other errors at the outset. DWP did not realise the security issues involved in offering claimants an interactive system. Too late, it attempted to address the challenge by retrofitting security into a legacy-style system, discovering the outcome to be hopelessly complicated for users.

At the same time, the first attempt to use new agile development techniques fell apart because the approach used – building up individual story lines – was highly inefficient. It was also badly undermined by its incompatibility with the central government clearance timetable, leading to subsequent – misplaced – accusations of lack of financial control from the National Audit Office.

The more rapid development schedule promised by agile also allowed an accelerated timetable. The failure to achieve this schedule was to be the direct source of much of the public criticism of the project. 

Five-way tussle

As the project’s problems escalated, a desperate five-way tussle developed within government, which played out through 2013. The Cabinet Office exerted direct control through the Major Projects Authority and its oversight of contracts. The Treasury, while aggressively forcing benefit cuts on the department, nevertheless proved supportive of its UC plans at this time thanks to the Liberal Democrat chief secretary to the Treasury, Danny Alexander. The prime minister, David Cameron, was nervously supportive and the Liberal Democrat deputy prime minister Nick Clegg much more positive.

The Cabinet Office was also ambitious to achieve its “digital by default” agenda and was explicit in wanting the initial system we had built scrapped and replaced by a new digital system built from the ground up. It put considerable resource into this redesign, led by the formidable Tom Loosemore, deputy director of the Government Digital Service (GDS). However, while GDS understood how the IT might work, it also suffered from ignorance – one that was the polar opposite of the department’s. It desperately under-estimated the complexity of a welfare system dispensing around £100bn a year. Accordingly, the initial timetable for the digital system started falling back at a rapid rate.

Read more about Universal Credit

The outcome of the tussles was the roll-out of two distinct systems, dubbed the “twin track”. The first system was shaped like a standard legacy benefit, albeit one in which claimants could make an application online. Thereafter, the interaction with DWP was through traditional post and telephone communication. Nevertheless, the department learned how the key operational elements should function through this system, as well as the behavioural response from claimants to the new arrangements.

Those findings helped shape the “full service” digital system which was developed and rolled out afterwards. It also maintained a sense of momentum at a critical time. Clearly, there were extra costs involved in this twin-track approach, although they were offset by the gains involved in putting people on the system earlier.

As is often the way in effective responses to crisis, however, the twin-track approach may well have been the best way to introduce a complex new system. The early version acted as a pathfinder to inform the structure of the full-service offering.

Lessons learned

The subsequent response of the department to this baptism of fire has seen a wholesale restructuring of its IT capability, pulling development and maintenance of systems in-house and integrating its services more closely with operations. DWP has built a genuinely agile capability in development. It has also given more authority to the operating teams, so that the detailed structuring of UC reflects the behavioural responses from the claimant base.

These changes represent a challenge to other government departments considering major developments. Purely in terms of IT, the lessons learned imply bringing development capability back in-house; building big integrated teams to adopt agile technology; and reversing a policy-led culture to one reliant on operational feedback.  

If departments do not rise to the challenge, there will be plenty more horror stories for Computer Weekly to lay bare.

Clashing Agendas book cover

David Freud was one of the architects of Universal Credit and was minister for welfare reform in the coalition government from 2010 to 2015. His book about the implementation of Universal Credit, Clashing Agendas: Inside the Welfare Trap, was published in June 2021.

Read more on IT project management

Data Center
Data Management