ex_flow - stock.adobe.com

Digital government: problems on the horizon

Are government’s digital programmes becoming the next Post Office Horizon scandal?

How time flies. Believe it or not, we’re now in the fourth decade of UK government ambitions to use technology to modernise and improve public administration and our national institutions.

And now, in 2024, it’s the turn of artificial intelligence (AI) to provide us with a miraculous “once-in-a-generation opportunity … to revolutionise our public services for the better and to deliver real, tangible, long-term results for our country.”

Meanwhile, back in the real world, ever since the UK’s first pan-government website launched in late 1994, we’ve seen the rather more mundane reality of departmental forms, processes, and information moved from paper onto a screen. Yet there’s a big difference between presenting departmental transactions in a web browser and the much more significant “digital transformation” of the public sector.

The National Audit Office (NAO) provides insight into the reasons for this long-running gap between political aspiration and reality. The spending watchdog observed that changes “have mainly focused on improvements to the citizen’s online experience ... Overall, the changes will have limited impact on the efficiency or effectiveness of service delivery … Without a deeper understanding of digital transformation, senior leaders often focus on tactical solutions and quick fixes and avoid addressing the underlying inefficiencies that contribute to driving future costs.”

The problems

The NAO’s library of reports highlights numerous problems over multiple decades, from the Home Office’s Digital Services at the Border to HM Revenue & Customs’ Making Tax Digital and the Government Digital Service’s flagship failure Gov.uk Verify.

Alongside the NAO’s compelling library of evidence, media and civil society reports reveal an interwoven and interdependent mix of policy and technical failures. They span everything from breaching the rule of law to the routine loss or alteration of data in mission critical national systems — a problem worryingly similar to the failed Post Office Horizon system.

Current examples include:

Unlike the Post Office’s Horizon system, where supplier Fujitsu can be held responsible and pursued for compensation, it’s less clear who to hold to account for unreliability and poor performance when digital programmes are owned and designed by in-house teams. Yet as some of these examples demonstrate, in-house design and delivery is not necessarily any better at creating successful outcomes than more traditional contracting and outsourcing.

The causes

Public sector transformation relies on having the right leadership and team experience, capabilities, and culture in place — regardless of whether delivery happens in-house or is outsourced under government oversight.

However, appropriate skills remain in short supply, leaving government's modernising ambitions at continuing risk of failure. This makes it even more surprising that government recently published proposals that would have reduced its access to technical expertise by deterring smaller suppliers from bidding for contracts. Sensibly, these proposals appear to have been dropped after a supplier backlash.

The gap between the longstanding political aspiration for a digital transformation of the public sector and the reality suggests an enduring organisational and cultural challenge

The current problems being reported with government programmes appear similar to those unearthed in 2017 by internal assurance reviews, which found widely variable approaches to the quality of programme design and delivery. These included teams lacking practical experience of managing and designing scalable, secure, resilient systems. Some teams were found to be checking and approving their own code into production systems without any meaningful quality assurance.

One system presented a highly praised user interface and was successfully demonstrated to an enthusiastic minister. But it was all “smoke and mirrors,” lacking connectivity to any other systems. As remedial work was carried out to integrate the interface to existing departmental networks, systems, processes, and data, everything ground to a halt.

Fundamental design and engineering flaws caused the solution to become slower and slower and eventually lock up and hang. For a system intended to support thousands of simultaneous users across the UK in near real time, it was not fit for purpose.

Time and resources were also pointlessly displaced into hand-baking bespoke “special” infrastructure rather than consuming commodity services and products. Tactical design decisions would pass small-scale unit testing, but create bottlenecks and poor performance as systems were scaled up to meet real world demand. Solutions would be signed off as fit to enter service without any attempt to ensure they were legally compliant and not in breach of the rule of law.

This latter problem reflects the widespread practice of policy and digital teams inhabiting largely separate worlds. Yet policy and technology are complex, dynamic, and interdependent: they need to be co-designed — managed, developed, and adapted together to reflect and respond to an ever-changing political, legal, social, and economic landscape.

Endlessly repeated political ambitions for technology





“[Technology will] provide better and more efficient services to businesses and to citizens, improve the efficiency and openness of government administration, and secure substantial cost savings for the taxpayer.”

Government Direct

Conservative administration


“Technology is revolutionising our lives … [It] offers huge scope for organising government activities in new, innovative and better ways.”

Modernising Government

Labour administration


“[Technology will allow] us to give citizens what they now demand: public services responsive to their needs and driven by them … it provides us with the means to deliver public services in a way that maintains their quality but brings down their cost.”

Putting the frontline first: smarter government

Labour administration


“[Technology will help us] deliver better public services for less cost … [It] will enable the delivery of public services in very different ways to the past.”

Government ICT Strategy

Coalition administration


“By harnessing digital to build and deliver services, the government can transform the relationship between citizen and state … putting more power in the hands of citizens and being more responsive to their needs … [we will] assemble services more quickly and at lower cost.”

Government Transformation Strategy

Conservative administration


“[We will] transform digital public services, deliver world-class digital technology and systems … Improving the way we use digital and data will enable the government to operate more efficiently.”

Transforming for a digital future

Conservative administration

Source: Digital Government and e-Government Archives

Problems of technical design adrift from policy and legal requirements aren’t a recent phenomenon. The Rural Payments Agency experienced similar issues in 2015 - the software locked up when multiple farmers (or even a single farmer using multiple browser windows) logged in. Legal deadlines were missed, fines were levied. Things got so bad that, according to the NAO, “in response to serious failings of the system, the online application system was withdrawn and replaced by ‘paper-assisted digital’ applications.”

The gap between the longstanding political aspiration for a digital transformation of the public sector and the reality suggests an enduring organisational and cultural challenge - a lack of capacity and capability in the leadership and management of large-scale policy initiatives and their associated design, development, and implementation. It’s a problem compounded by the absence of independent policy, legal, and technical assurance.

The fix

There’s insufficient data available to assess definitively whether in-house developed systems cause more, fewer, or the same level of problems as those implemented by external providers. But as the number of high-profile problems and failures make clear, without the right capabilities in place, bringing systems development in-house isn’t in itself a magical panacea.

Just as the forensic accounts Second Sight identified issues with Horizon, government needs an independent function to establish objective and ongoing policy, technical, and legal assurance of public sector programmes. It will need teeth to succeed, together with improved access to meaningful information to help analyse and identify programmes going adrift, or at risk of going adrift, to help flag and fix them quickly and effectively. And it needs the authority to terminate “no hope” programmes in a much more timely and effective way, rather than letting them limp on, undermining governments’ policy aspirations and bleeding precious public resources and credibility.

Although it may not be as glamorous and eye-catching as suggesting AI will somehow “revolutionise our public services for the better,” a cross-government, independent assurance function is essential to help support and reset digital government onto a better, and more successful, track. It will provide essential, baseline insight, exposing what’s going well and what isn’t, and helping inform decisions on where interventions are required to improve organisational culture, leadership, and skills to ensure better and more consistent results.

And, who knows? It might also prove an important step towards helping politicians deliver on their nearly 30 year old vision to use technology to modernise policy making, update our public institutions, and provide better outcomes for citizens and businesses.

Read more about digital government

Read more on IT for government and public sector

Data Center
Data Management