The Bank of England is exploring the use of kill switches that could halt trading in the event of artificial intelligence (AI) models going astray, as it accepts existing regulatory frameworks may not be adequate.
“What these two examples – agentic commerce and agentic trading – both highlight is that, as AI capabilities increase, we must keep asking whether existing, technology-agnostic regulatory frameworks remain sufficient,” she told the audience.
Breedon said financial firms will not be able to rely on a guarantee of human oversight. “Our frameworks were not built to contemplate autonomous agents, and relying on a human in the loop for all agent actions is unlikely to be realistic. More sophisticated governance and accountability frameworks may be needed,” she added.
Breeden said that, from late 2024, AI systems were trained to reason through requests, but today they can autonomously chain together sequences of actions.
This has huge implications for regulating trading and commerce, and protecting against cyber attacks. “The financial system looks likely, therefore, to evolve quickly into one that operates more autonomously, at scale and speed: AI agents transacting on behalf of consumers and merchants; devising and executing trading strategies in financial markets; and identifying and chaining together cyber vulnerabilities,” she said.
What if autonomous agents go rogue?
In trading, Breeden said firms currently use autonomous AI for lower-risk operational tasks, such as research, but added that this could change quickly.
In a financial system that operates more autonomously, at scale and speed, she said AI agents could devise and execute trading strategies.
Our frameworks were not built to contemplate autonomous agents, and relying on a human in the loop for all agent actions is unlikely to be realistic. More sophisticated governance and accountability frameworks may be needed
Sarah Breeden, Bank of England
“If AI agents respond similarly to the same prompts or triggers, they could amplify volatility in stress – especially if their objectives drift from original goals or public policy objectives, in a manifestation of the misalignment problem that can arise with some AI models.”
She asked whether firms can “observe and contain their resulting behaviours”.
“We are experimenting with the Bank of International Settlement Innovation Hub and the Bundesbank on simulation methods to understand which aspects of agent design could drive herding behaviour,” she added.
Breeden said the organisations are also considering whether there is a need for guardrails, similar to circuitbreakers or kill switches, that would limit or stop trading market-wide if faulty AI models cause market meltdown.
MPs on the Treasury Committee reported that the position adopted by the Bank of England and the FCA, which they described as a “wait-and-see approach”, was exposing the public to “potential serious harm”.
Cyber weakness a pressing concern
Breeden’s most pressing stability concern is a step change in agentic AI’s cyber capabilities, which includes the ability to identify cyber vulnerabilities en masse.
“In the hands of defenders, these tools strengthen cyber resilience. But in malicious hands, they materially increase the chance of attacks that could harm financial stability,” Breeden warned.
She said the main challenge for regulators is maintaining the advantage of cyber defenders as malicious actor capabilities evolve. She added that this is particularly the case “in an increasingly severe cyber threat environment, including from geopolitics and ransomware”.
Breeden added that for financial stability, vulnerability patching must happen quickly – across the financial sector, key third-party technology providers and the wider national infrastructure.
What disruption response might look like
With the increased risk of mass disruption in the sector, Breeden said the regulators should consider whether the firms they regulate require “enhanced recovery options for core systems”.
She said cooperation between financial services firms could be an option, with, for example, other banks providing basic functions to the customers of banks that are unable to provide them. “In a cyber context, do we need systems that allow one institution to pick up another’s basic functions during disruption, as in Ukraine’s Power Banking programme launched in 2022?” This initiative was created to guarantee financial resilience during blackouts and grid disruptions.
Breeden added that another consideration is whether “key firms” could be made to have completely separate failover capabilities, or to be able to rebuild compromised core systems quickly from “bare metal”.
UK financial services regulator wants to increase understanding of AI’s benefits and how firms are managing the risks it poses, as take-up grows and use cases increase.
