Avoid expensive AI agents with these five design imperatives

1. Stop moving data to AI

Clarke’s first imperative addressed the scattered state of enterprise data. He noted that while modern AI runs on data, most organisations are grappling with data silos and unprepared for agentic use cases.

“In most enterprises, the data is scattered across dozens of systems,” Clarke said. “Eighty to ninety percent of it is unstructured – none of it is connected in a way to effectively power agents at scale.”

Rather than copying and shifting massive datasets to external cloud-based models, Clarke argued that organisations must bring the models to their existing storage.

“To power agents at scale, you need a real-time connected knowledge line, and here’s the structural and architectural decision you’re going to have to make: don’t move the data to the AI, move AI to the data,” he said. “That’s fundamentally a different approach and, quite frankly, a decision that we need to make now.”

2. Design for massive, multi-step inference workloads

While early corporate AI efforts focused on centralised model training, Clarke highlighted that the exponential growth in AI inferencing demands an entirely different tier of compute.

Unlike simple chatbots, agentic AI systems perform multi-step reasoning, frequently calling various models to plan, execute, and iterate on complex workflows.

“When I think about inference, you have reasoning models executing multi-step chains,” Clarke explained. “These workloads are 10 to 100 times – some even say 1,000 times – more compute-intensive than what we were running just 18 weeks ago. An AI-native enterprise has to be built for both [training and inference].”

3. Demand a ‘receipt’ for every autonomous action

Because AI agents do not just retrieve information but actively execute business processes – such as modifying customer records, placing orders or issuing financial transactions – security can no longer be treated as a passive layer.

Clarke called for enterprise systems to be redesigned to log, track and reversibly commit every action taken by an AI agent.

In an AI workforce, every action needs a receipt. That’s not a compliance checkbox; that’s how you build trust into the system that will act on its own

“They don’t just call a model; they call your CRM [customer relationship management], ERP [enterprise resource planning] and financial systems, and they call your customer databases,” he said. “Every one of those touch points has to be secured, logged and reversible.”

He noted that tracking agentic actions is key for business continuity: “When an agent acts on your behalf – changes prices, updates a customer record or initiates a return workflow – you need to know what it did, why it did it and how to undo it if it got it wrong. In an AI workforce, every action needs a receipt. That’s not a compliance checkbox; that’s how you build trust into the system that will act on its own.”

4. Integrate the enterprise stack

Clarke warned that if an organisation’s existing software stack is not unified through application programming interface (API) orchestration, any deployed AI agents will become costly, isolated failures.

“The agent becomes the coordinator,” he said. “Agents need to plan tasks, call tools, execute work and handle exceptions across your entire stack. That means an API code architecture, workflow orchestration, and an agent framework that can do multi-step execution.”

Without this level of deep, stack-wide integration, Clarke delivered a blunt warning for CIOs: “If your stack can’t do that, your agents are going to be siloed and expensive, and that’s simply not a conversation you want to have with the board.”

5. Master the economics of token routing

Finally, Clarke urged IT leaders to optimise their tokenomics rather than defaulting to the most advanced frontier models for every task. True efficiency lies in routing the right task to the most cost-appropriate model, whether it sits on-premises, at the edge, or in the cloud.

“The question isn’t whether consumption grows – it absolutely will. The question is, are you running the right tokens on the right infrastructure?” Clarke said, adding that generating a routine summary, for example, doesn’t require the use of a frontier model.

“If you do that correctly, you’re going to get optimal performance, privacy and cost efficiency. Run everything in one place because it’s easy, and you’d be ready for a surprise – a large bill that’s only going to get larger. The notion of token routing – where to put that token – is going to be one of the most important decisions we will make,” Clarke concluded.