How not to go off the rails with agentic AI

Early days adoption

So far, few examples are running in production. For many organisations, the agentic AI hasn’t yet gone beyond a conversation.

Consider that AI agents may input AI model outputs in another and be working towards a goal without full oversight. According to IBM’s video series on AI agents, guardrails must be considered at model, tooling and orchestration layers.

Peter van der Putten, AI lab head at workflow automation specialist Pegasystems, says that many organisations do not feel they will mitigate agentic challenges such as governance and risk this year or next. “Some go, ‘They can’t even pass a Captcha.’ Then you have the believers saying, ‘Create as many agents as you want and let them run amok.’ Both views are flawed,” he says.

Start with selected single-agent use cases, see how well they perform, and ground agents in your enterprise architecture artefacts, workflows, business rules, appropriate context and user access and so on.

Then contrast with reality – are the agents doing the right thing and are they achieving their goals? Those are the kinds of strategies a business might apply to enable agentic AI.

“Throw in a bunch of requirements, use process mining to see the actual process (versus what people tell you the process should be). Clean that up, put in other requirements and then give that as input into more like design agents that can help you,” van der Putten says.

Then the human is in the loop because you can see what you agree with or not. Only then do you build an application that can run very predictably at run time. Of course, if you can’t “automate things away” and need human oversight of everything, agents might not be the right answer, van der Putten adds.

Choose the right agents or LLMs for each aspect and build on that. In insurance, one agent might assess risks, another claims, while yet another interacts with other employees or even an end customer. And then, is a sales-focused agent the right answer in that circumstance? That also depends – you need the exact agent for the context.

Afterward, an agent layered on top might operate by “understanding” the individual steps or specific workflows to call – or not – in a given situation; one right at the end might check previous work. And when you hit a roadblock, you “escalate back to the human”.

Only down the track might you consider layering multi-agent systems on top where specialised agents for particular tasks talk to each other.

Van der Putten says: “The tools need clear processes, rules, policies and maybe non-generative predictive models that assess likelihood of fraud or similar. Pull the context, get a full picture of the situation and the request.”

Measuring the benefits

Think about it as slightly smarter robotic process automation (RPA), says Simon James, data strategy and AI managing director at Publicis Sapient. Start with mapping processes and determining which might benefit from AI agents versus human judgement or traditional automation. Devising a clear, compliant framework can help.

The more choices you incorporate, the more scope the AI has for things to simply go wrong and the more difficult it becomes to govern. “There’s a wheel of death going on somewhere while several agents are talking to one another, even in highly optimised machine-readable code, and not in English, adding latency to 20 systems,” James adds.

Because agentic AI is so new and people often don’t have the skills, industry is still figuring things out. Maybe it can run three different routines or functions and it’s got choice between them, but there’s not much choice there, James warns. “And it’s about how a Salesforce version, for example, connects to ERP or CRM or whatever else so they can pass the logic between each other and the handoff isn’t painful.”

Dominic Wellington, AI and data product marketing director at platform Snaplogic, reiterates that many people are still figuring things out “the hard way” in agentic AI: “Lawyers and compliance are getting involved, and can ask tough questions before sign-off on going into production. It’s so easy to stand something up that looks cool, but we see horrendous drop-off rates. Half to 80% of projects never make it to production.”

Often the subset of information that powers the pilot to success will not work writ large. When you want to connect to “crown jewels” – such as the corporate database or CRM – you may need to reconsider access to that data and more complete enforcement of policy and practice.

“If you’re AstraZeneca, for example, you don’t want your pharma pipeline winding up in some model’s training data,” he says. “And having ‘ground truth’ is critical. I never have to go back more than a couple of days in my news feed to see an instance of a lawyer having cited non-existent precedent because they asked ChatGPT – and it’s not just lawyers.”

Of course, with retrieval augmented generation (RAG), for example, you can vectorise appropriate information into the data store, with the LLM responding based on what’s in a particular data store, offering control over what it sees or can respond with. With data masking, Quality of Service (QoS) and role-based access control you can go far, Wellington agrees.

That said, considerations run the gamut from ethical challenges to compounding errors, security risk, scalability, explainability and accountability and bias to privacy to, quite simply, the potential for unintended consequences. Agentic AI needs transparency, but it’s not easy to know how.

This all sounds familiar to early-days cloud adoption – but with AI, the cycle from hype to disillusionment has accelerated. However, there are early adopters that can be learned from. “It can be the quieter second wave that actually points the way,” Wellington adds.

Sunil Agrawal, chief information security officer (CISO) at AI platform Glean, says it’s worth the fight. AI agents can reshape how work is done, helping to surface and make sense of needed data. But scaling these systems securely and responsibly is critical.

Agents must respect user roles and data governance policies from day one, especially in highly regulated environments, and observability of what’s going on is crucial. This covers what data they access, how they reason and which models they rely on.

“AI agents are only as reliable as the data they’re built on,” Agrawal says. “Ground them in accurate, unified internal knowledge. And threats like prompt injection, jailbreaking and model manipulation require dedicated defences. A strong governance framework helps ensure agents operate safely, ethically and aligned with organisational policy.”