Agentic AI is driving rethink of enterprise architecture and tokenomics

Busting the GPU training myth

While the initial AI boom had fuelled the rush to secure graphics processing units (GPUs) for model training, Roese said the infrastructure requirements of enterprises are vastly different from that of hyperscalers.

“The myth out there is that enterprises need thousands of GPUs,” said Roese. “Our biggest workload inside of Dell only sits on 16 GPUs and supports 40,000 people. You don’t need thousands of GPUs in an enterprise, because for each workload, agent or project, you only need a handful of GPUs, sometimes half a GPU.”

That’s because much of the enterprise AI estate is entirely focused on inference, not training. “For agents, you only need inference. There’s no training for agents.”

That said, the architecture needed for inference workloads is changing as well. When enterprises were building chatbots, the architecture resulted in a very light CPU load. AI agents, however, use external tools, communication protocols, and knowledge graphs – components that do not naturally live in the GPU.

“When you move to agentic, it’s almost balanced,” said Roese. “The number of CPUs and GPUs are very similar, about maybe for every two GPUs you have a CPU. You don’t just build an AI infrastructure with a pile of GPUs – you build it with GPUs and traditional CPU compute.”

Air-gapped frontier models and the edge

Enterprises are also benefiting from changes in how powerful AI models are being deployed. A year ago, the most capable frontier models were locked behind cloud APIs (application programming interfaces).

Our biggest workload inside of Dell only sits on 16 GPUs and supports 40,000 people. You don’t need thousands of GPUs in an enterprise, because for each workload, agent or project, you only need a handful of GPUs, sometimes half a GPU

But with hyperscalers now enabling top-tier models to be run on-premises through services like Google Distributed Cloud, Roese noted that a private model can now be deployed in multiple topologies. “You can consume it in a virtual private cloud or your datacentre, and you can air-gap it from everything else. We didn’t have any of those options, except the API one, a year ago,” he said.

Simultaneously, AI is moving to the edge in a structured way. Roese pointed to the recent emergence of agentic frameworks like OpenClaw that run natively on devices and AI PCs. “Those have finally put some structure around running agents on devices, and that’s incredibly powerful and not a fad,” he said.

Re-architecting the data layer

Meanwhile, data strategies are evolving in tandem with agentic AI developments. Roese warned that bolting standard data storage systems onto AI compute clusters is no longer enough to meet the performance demands of AI agents.

Instead, organisations need to build knowledge and context layers comprising vector databases, graph databases, and data annotation tools. These layers cannot sit isolated and must be deeply integrated into compute.

“One of the performance bottlenecks is you can’t get data fast enough to the GPUs to do the work,” said Roese, adding that “the GPUs you’re paying for are sitting idle, waiting for data.”

To reduce this latency, he said Dell’s AI data platform is now plumbed into Nvidia’s Cuda-X interfaces, effectively running data layer services directly at GPU speed.

Mastering tokenomics and model routing

With more model deployment options available at different pricing mechanisms, IT leaders will also have to manage the cost of AI consumption – even as the cost per token is expected to decline over time. Because “there’s no path where it becomes cheaper to do AI,” enterprises must treat AI workloads as an arbitrage game, said Roese.

Using specification-driven development – where AI writes software based on a markdown document – as an example, he noted that if an agentic framework spawns dozens of coding tasks and blindly sends them to top-tier models, enterprises could end up with a higher bill.

But with model routing, enterprises can ensure complex planning tasks, such as creating software specifications, are sent to expensive frontier models, while routine coding tasks are routed to smaller, on-premises open-source models where energy is the only operational cost.

“Building a piece of software and doing spec-driven development might have four or five different economic paths to ultimately get to the best overall economic efficiency,” said Roese. Mastering model routing, he added, will be a competitive differentiator and helps lower the cost of product development.

The human element

Ultimately, the hardest part of operationalising agentic AI relates to the human element. Roese described the traditional human job as a “container of work” that includes a mix of hygiene, productivity, coordination, and expert tasks. Agents cannot perform an entire job, but they are highly capable of executing specific types of work within that container.

Dell itself had audited 6,400 jobs across its own business to see how AI agents would impact its workforce.

“The first thing we realised is every single job in the company is going to change,” said Roese. “I’m taking work out of the job and removing stuff from the container. If the container is now only half full, do I need half the number of people, or do I expand that by half? Am I able to do more expert work?”

Indeed, the impact of AI on the workplace is so profound that change management has become a key remit of IT leadership.

“For the last four months, I’ve spent 50% of my time dealing with human dynamics,” said Roese. “AI has ceased being a technology and an ROI [return on investment] discussion. It’s now very much an organisational and human dynamic discussion. You simply can’t use these things unless you fully understand how you’re going to adapt the human population around them.”