Muhammad - stock.adobe.com

News

AI safety cannot wait for a ‘Chernobyl moment’, experts warn

As AI becomes increasingly capable, tech leaders at Singapore’s ATxSummit urge governments and industry to build safety and accountability into AI systems before a major disaster strikes

Ai Lei Tao
By
Published: 25 May 2026 10:28

The global debate on artificial intelligence (AI) governance is entering a more urgent phase as systems become more capable, harder to evaluate and increasingly embedded in daily life, tech leaders and experts said at the recent ATxSummit tech conference in Singapore.

Panellists said the question is no longer whether AI should be governed, but how quickly governments, industry and society can build systems of accountability that can keep pace with the technology.

Waiting for a major AI-related disaster before acting would be a serious mistake, warned Stuart Russell, distinguished professor of computer science at the University of California, Berkeley. He drew a comparison with the Chernobyl nuclear accident, saying that “without safety, there are no benefits”.

“If there is a Chernobyl-scale disaster with AI, it's not just going to be a regulatory response; it's going to be a societal response. People will say, ‘shut it down’. All of those trillions of dollars that we hear about being invested will be wasted,” he said.

That urgency was echoed by Karan Bhatia, global head of government affairs and public policy at Google. He suggested the need for a revolution in how government and industry work together to face these challenges.

“The technology is moving far too fast for traditional methods of governance to be applicable,” said Bhatia. He called for “a constant, regular level of interaction going on between regulators and industry – everything from identifying trends in threats and opportunities, intel sharing on a constant basis, to steady iteration on what the regulatory possibilities might be”.

For Elham Tabassi, director of the AI and Emerging Tech Initiative at the Brookings Institution, the answer is to build AI governance into the development process from the start, ensuring systems are trustworthy by design.

“We cannot keep treating governance as something we check only after an AI system is already built or released. Governance must be part of the design, development, deployment and monitoring process,” she said.

Practical safety steps

Even though AI governance is still lagging behind the technology, there are practical safety steps that governments can take immediately, said Ya-Qin Zhang, chair professor of AI science and founding dean of the Institute for AI Industry Research at Tsinghua University.

He said AI governance can learn from safety practices in the aviation, nuclear power and pharmaceutical industries. He pointed to measures such as labelling AI-generated content, registering AI agents and preventing uncontrolled agent self-replication.

Russell added that AI governance should follow the same basic principles used in sectors such as medicine, aviation and nuclear power, with “the onus on the developer” to provide evidence that their systems are safe enough for use.

Current AI evaluation methods are struggling to keep up with the technology, according to Tabassi.

“The evaluation basis is thin,” said Tabassi, noting that the evidence from current AI testing is not deep or reliable enough. While pre-release testing remains important, she warned that benchmarks do not always predict how AI systems will behave in real-world settings, especially when models and agents may behave differently during tests.

She argued that AI governance must move from one-time certification to continuous evidence-gathering.

“Pre-release testing and pre-deployment testing are important, but that type of evidence needs to continue via continual monitoring of the systems post-deployment, incident reporting, and observing behaviour in the wild rather than just in the laboratory,” said Tabassi.

Rebecca Finlay, CEO of the Partnership on AI, agreed that testing AI before release is important, but is not enough. There is a need to understand what happens after AI is used in the real world.

While she noted some progress in areas such as usage data and labour market impact analysis, she warned that incident reporting and environmental disclosures remain difficult to compare without common standards. Greater transparency, she argued, must be matched by clearer measurement frameworks so that policymakers, companies and the public can understand AI’s real-world effects.

New challenges with agentic AI

Zhang pointed out that many current evaluation methods are no longer useful as the technology moves from generative AI to agentic AI, because “previously, most of the research tools and evaluation were optimised for pre-training”.

With complex, long-range capabilities, he said an agent can autonomously implement thousands of steps over 20 or 30 hours, making testing more difficult because “everything is dynamic”.

Tabassi agreed that agentic AI cannot be evaluated in the same way as traditional language models, as it presents a far more complex governance challenge.

“Agentic AI and agents act, plan, orchestrate, and then operate in an environment where the environment itself changes in reaction to them,” said Tabassi. In contrast, she noted that large language models (LLMs) can usually be evaluated simply by comparing inputs and outputs.

She warned that agents may also behave differently when they know they are being tested, making their real behaviour harder to measure.

Finlay said organisations need clearer ways to determine when AI agents should be monitored and at what level. She said companies can begin by assessing three factors: the stakes of the task, whether the agent’s actions can be reversed, and what access or permissions the agent has been given.

Bhatia noted that AI governance is difficult because AI is global, and different countries may set different rules. He warned that if countries adopt very different guardrails, companies may shift activity to jurisdictions with more favourable rules. While he supported global convergence around shared standards, he said each country will likely balance risk and innovation differently as they compete to attract AI investment and development.

“Companies should begin with lower-risk pilots before moving into more high-risk, multi-agent deployments,” said Finlay.

For Russell, the message was short and direct: “Don’t wait for Chernobyl… Take steps now before it’s too late.”

Read more about AI in APAC

Read more on IT governance