Agentic AI to drive heavy infrastructure demands

Organisations that are exploring the capabilities of artificial intelligence (AI) agents must prepare for a substantial increase in infrastructure demands that will surpass current generative AI (GenAI) requirements, according to Soo Mei May, chief of AI global solution specialist at Dell Technologies.

In a recent interview with Computer Weekly, Soo noted that while early implementations of agentic AI are likely “straightforward ones” that will not require reinforcement learning, expanding its use across multiple business functions will require more than the existing GenAI infrastructure that handles single-function agentic AI experiments.

“With GenAI, you put in input tokens and get output tokens,” Soo explained. “But with agentic AI, the number of tokens will multiply by 20 to 30 times, which means more compute power is involved.”

Beyond compute, memory and storage requirements will also increase. “In agentic AI, you’ll need persistent, long-term memory to hold all the past conversations, so storage requirements are going to be very heavy and you’ll need to retain this data for the next three to five years,” Soo said.

This increased storage demand isn’t just for conversational data, it’s also used for governance. “The autonomous nature of agents makes it even more urgent to build transparency and explainability into the whole system,” Soo said. “We need to consistently log everything so we can explain why an agent came to a certain conclusion, for example, which means storage requirements, again, are going to be higher.”

Networking infrastructure also faces a significant challenge with agentic AI systems often involving multiple agents interacting with each other and retrieving data from internal and external sources simultaneously. “Latency definitely plays a very important role here,” Soo said. “If an agent takes five minutes to find information, that will not work. So, the networking piece has to be very robust.”

While many organisations, particularly those in financial services, manufacturing and healthcare, have already built up substantial on-premise or cloud-based GenAI capabilities, agentic AI will require further investment to extend its use to more business functions, Soo said.

In terms of large language model (LLM) deployment to support agentic AI applications, Soo said organisations will need an orchestrator model with a minimum of 70 billion parameters for it to plan and orchestrate AI agents well, as well as make coherent decisions.

She added that small language models, which perform better on narrow, focused tasks, can be used to support the LLM, which can also call upon machine learning models for specific predictive tasks.

Dell is working to prepare customers and partners for the shift to agentic AI through clinics, workshops and proofs of concept. But the primary challenge, Soo believes, is not the technical skills gap, but what organisations can do with agentic AI and understanding what it is.

“They probably think it’s an LLM and not many know it’s a system with many things – it’s not just your LLM; you’d also need an orchestrator, a prompt system, persistent memory, tool calling, and interfaces with agents and users,” she said. “But I believe they’re getting better because they are taking time to learn more about it.”