Hoda Bogdan - stock.adobe.com
Is there no stopping the AI spending spree?
Looking at Nvidia’s latest financial results, it would seem that spending on compute is set to increase tenfold by 2030
Expect datacentre spending to increase tenfold. That was among the claims Nvidia CEO Jensen Huang made during the company’s latest quarterly earnings call. He forecast that capital expenditure (CapEx) on datacentres would increase from the $300-400bn mark today to $3-4tn by 2030.
Huang’s remarks at the end of February came a few days after Microsoft’s AI Tour London event, when CEO Satya Nadella effectively called for enterprise software developers to use the capabilities now built into Microsoft 365 to create agentic AI workflows for streamlining business processes.
Nadella discussed the need to have an efficient token factory, where phrases or tokens can be streamed into AI engines that interpret natural language for querying large language models (LLMs). The Microsoft vision of enterprise AI is built on the M365 foundation, which acts as a knowledge store on which a new category of knowledge-based software can be built.
During his keynote presentation, Nadella spoke about the intelligence that exists in the various IT systems used across the business. He said that businesses should be able to harness the intelligence that already exists enterprise-wide, starting with what he described as the “data underneath Microsoft 365”, which, according to Nadella, represents the people in the business, their relationship to coworkers, and work artefacts such as projects, calendars and communications data. “This is massive information,” he said, which can be used to bootstrap agentic AI projects.
“Our goal is to have all of the innovation and the systems available in the token factory,” said Nadella. “That way you can build software which has the ability to use all of the capability [we provide] to train models and deliver models for inference.”
In effect, he sees the Windows software developer ecosystem evolving to where it is now a Microsoft 365 ecosystem, where enterprise data is stored in AI-enabled office productivity tools such as Word, Excel, PowerPoint, Teams and Outlook, and these can be used as the foundation for a new generation of applications that can draw on these AI knowledge sources.
It is this idea that all software will need to be knowledge-aware, which Huang spoke about during the company’s earnings call. “Token generation is at the centre of almost everything that relates to software in the future and relates to computing,” he said. “If you look at the way we use computing in the past, however, the amount of computation demand for software in the past is a tiny fraction of what is necessary in the future.”
Read more AI hardware stories
- Nvidia unveils Vera Rubin architecture to power AI agents: The AI chip giant has taken the wraps off its latest compute platform designed for test-time scaling and reasoning models, alongside a slew of open source models for robotics and autonomous driving.
- Microsoft Maia 200 AI chip could boost cloud GPU supply: Industry watchers predict ancillary effects for enterprise cloud buyers from Microsoft's AI accelerator launch this week, from GPU availability to Nvidia disruption.
According to Huang, the amount of computation necessary to run AI is 1,000 times higher than the computing power needed to run non-AI software. “The computing demand is just a lot higher,” he said. “And so, if we continue to believe there’s value in it, then the world will invest to produce that token.”
When asked whether Nvidia is confident that its customers will continue to have the ability to spend more on AI infrastructure, which could impact Nvidia’s ability to grow, Huang spoke about the opportunity in enterprises to make use of agentic AI and its widespread usefulness across organisations.
“We have now seen the inflection of agentic AI, and the usefulness of agents across the world and enterprises everywhere,” he said. “You’re seeing incredible compute demand because of it. In this new world of AI, compute is revenues. Without compute, there’s no way to generate tokens. Without tokens, there’s no way to grow revenues.”
At least, that is how he positioned AI for the investment bank analysts on the earnings call. The company posted fourth quarter revenue of $68bn, up 73% year-over-year. Datacentre revenue increased by 75% to $62bn, which Nvidia said was being driven by demand for its Blackwell architecture and AI inference deployments. It also reported networking revenue of $1bn, up 3.5x year-over-year, fuelled by adoption of NVLink, Spectrum X and other Nvidia ethernet technologies.
Last year, during his keynote presentation at the GTC conference in the US, Huang claimed that the lowest cost per token was being achieved using the most expensive GPU – which at the time was the Grace Blackwell NVLink 72.
Nvidia describes the GB200 Grace Blackwell as a “superchip”, which connects two high-performance Nvidia Blackwell Tensor Core GPUs and the Nvidia Grace CPU with the NVLink-Chip-to-Chip (C2C) interface, capable of delivering 900 GBytes/s of bidirectional bandwidth.
Significantly, the architecture means that applications have coherent access to a unified memory space. According to Nvidia, this simplifies programming and supports the larger memory needs of trillion-parameter LLMs, transformer models for multimodal tasks, models for large-scale simulations, and generative models for 3D data.
‘Huang’s Law’
Some industry observers have coined the term “Huang’s Law” to describe his perspective of how each new generation of GPU delivers a 10x increase in performance, compared with Moore’s Law’s doubling of performance every 18 months.
Nadella and Huang both spoke about how newer hardware is more energy-efficient at running AI workloads. During the Microsoft AI tour, Nadella noted that today’s system supports an entirely different memory hierarchy, which he said means “there’s now no latency with AI inference”.
The messaging from both the Microsoft and Nvidia chiefs is that the best efficiency is achieved by taking advantage of the capabilities available in these new systems. “There’s an unbelievable renaissance happening with these systems and workloads, whether they’re training workloads or inference workloads, they are unlike anything we’ve seen in the past,” said Nadella.
The tech sector is dead set on getting enterprises to adopt more and more AI. It is being built into knowledge-aware enterprise software likely to draw on the capabilities available in the newest generation of AI acceleration hardware.
Clearly, the business models of Microsoft and Nvidia are tied to increased demand for AI. But it is also apparent that the cost of deploying advanced AI systems is not going to get any cheaper. If anything, capital expenditure on datacentres will continue to increase at a phenomenal rate, fuelled by demand for these new AI systems and the AI acceleration hardware they need.
