Could Data Analytics and AI be used to reset the Election Agenda around Policies that will work?

UK has lost confidence in the competence and probity of  its institutions , from Press to Government (Westminster, Holyrood, Cardiff, Whitehall, Townhall)  That has serious implications for those competing for support during the longest election campaign in modern history.

It sounds politically very boring, but could short-order training for Officials and Policy Advisors in the use of Data Analytics and AI to transform policy formation and implementation, beginning with citizen-facing services like the tax and benefit system, help rebuild confidence around quick wins for Ministers and Mayors whose staff and supporter move rapidly to harvest the low-hanging fruit, while condemning those who stand in the way to oblivion?

Can what Donald Michie, (one of the god-fathers of Machine Intelligence) called “Knowledge Refining” be used to unravel “Fake News” (including unworkable policies) and identify, report and/or begin successfully removing the obstacles to, for example, getting supposedly unfit millions back into work, living in homes that are fit for the 21st century?

The Natural Language Systems on which Donald was working when he died have now merged with Digital Neural Networks to power Generative Artificial Intelligence services capable of delivering, at low cost the vision Donald described in 1982, at a seminar for the UK technical press (published as Intelligent Systems: The Unprecedented Opportunity). That vision included the use of Machine Learning (aka Generative AI) to analyse large volumes of knowledge (including manuals and rule books) to identify its provenance, weed out errors and falsehood and provide authoritative guidance to humans, whether or not they had access to a computer.

The example he used was the opening moves of chess, where the data used to train the early chess playing games (on which he had worked with Alan Turing) had been distilled into six basic principles that could be summarised in under 250 words. He looked forward to similar processes being used to streamline Government, beginning with removing the many nonsenses and irrationalities that had built up over the decades in the UK Tax and Benefit systems, and to politicians modelling the effect of new legislation before they approved it.

But the late 1980s saw the end of one several false dawns with regard to the future of AI. Ministers and their in-house IT teams had wanted to use computerisation to rebuild the tax and benefits system round “the customer journey” but the obstacles (from the lack of the relevant skills to overt and covert obstruction and sabotage from vested interests) were too great. The in-house reformers ran out of steam. Instead we had the rise of outsourcing and the hollowing out of the civil service as the best were head-hunted by suppliers to deliver the generation inflexible lawyer specified PFI contracts which are now coming to an end, leaving a legacy of bankrupt schools and hospitals.

In 2007 -8, when a team of MPs (with c9llective experience across a dozen Select Committees) looked at the Labour  Government  Transformation Agenda they found DWP was proud of its (finally live and outsourced) systems. These collectively handled over 1200 benefit rules. But barely 12 rules accounted for over 80% of claims. Most staff were unaware of most of schemes that might be relevant to claimants. The benefits received varied according to the start point for the claim rather than the needs of the claimant. And those in most need were commonly stuck in poverty traps, better off on benefits than in work.

The attempt to begin again with Universal Benefits soon got bogged down in the mud of the inherited outsource contracts. Then came Covid, with officials working from home over low bandwidth conferencing systems removing controls to enable those trapped in lockdown to survive. Waste, fraud, inefficiency and cost consequently mushroomed. Access to public services plummeted and has yet to fully recover. So too has access to well-researched uncensored news, as opposed to algorithm-driven rumour, picked up and magnified as supposed reality.

Now that the Civil Service is beginning to rebuild its in-house skills, after the zenith of outsourcing,  I would like to think that the disciplined (as opposed to imaginative) use of Data Analytics and Generative AI offers a way to using them to help rebuild belief in the competence of Central Government by moving rapidly to streamline the implementation of current policy and focus policy debate on that which is likely to work.  

The ability of the latest language processing systems to trawl rapidly through vast amounts of digitised legislation, including statutory guidance, means that Brexit could probably have been delivered to time, had officials been trained in its use and been working collectively in teams with high speed access to systems and colleagues.

But that means training officials and policy-maker to use the techniques and technologies to greatly improve their own efficiency (as “mind extensions”), not delegating the work to outsiders or “experts” incapable of  quickly spotting when AI is “hallucinating” because its wellsprings of Data Analytics have been poisoned, accidentally or deliberately, by bad data.

That means modular update programmes for those currently in post.

Hence one of the core areas of focus for the skills partnerships that I would like to see come out of the recent DPA APPG Round Table , see also Will 2024 be the year AI begins to transform Education, Recruitment, Training from cradle to dotage?

But the Civil Service will have to move rapidly to train in its own because the skills are not available on the open market, even if those with the skills understand the applications. Meanwhile demand for similar skillsets to support action on resilience and counter-fraud takes off . Once again the “solution” is to cross train those currently in post. In the case of resilience and counter fraud that probably means administrative accountants, auditors and compliance staff.  It may also mean that salaries for those with criminology (including data analytics) degrees will rise above those with cyber security degrees without similar AI and Data Analytics components.