How the AI boom is reshaping tech cost management

The hidden costs of homegrown AI

While off-the-shelf AI services offer convenience, building homegrown AI can be significantly more expensive. It requires securing highly coveted graphics processing units (GPUs) in the datacentre or the cloud, and addressing what Pinter calls “the hidden cost of AI”.

“It gets a lot more complex because now you’re talking about the infrastructure to support homegrown AI solutions,” he said. “If they are in the datacentre, then you need to consider the electricity costs to power these systems.”

Increasingly, the environmental footprint of AI is tying FinOps to GreenOps, particularly in the Asia-Pacific region where new climate laws mandate companies to measure and reduce carbon emissions. By optimising cloud usage, organisations can simultaneously lower their bills and carbon footprints.

Beyond public cloud services, nearly half of FinOps teams are actively managing physical datacentre costs to capture the full footprint of AI computing demands, according to the FinOps Foundation report. These teams are also working with environmental, social and governance (ESG) teams on sustainability initiatives.

The search for ROI

Despite significant investments in AI, many companies struggle to articulate its return on investment (ROI). “Many customers are missing that right now,” Pinter said. “They’ve been told, ‘Go do AI’, but they don’t have a clear end state in mind.”

With just 7.5% of enterprises baking FinOps into AI projects, according to IDC, practitioners are encouraging more businesses to calculate the exact unit economics of AI. For example, a bank that processes home loans could establish a baseline cost, say, $8 per loan for 1,000 loans a month, and measure the financial impact of AI implementation.

“Ideally with AI, you should see the number of mortgages increase and the processing time decrease,” Pinter said. “You could say, ‘We’ve tripled that and lowered our unit cost by 10%.’”

This is where the Technology Business Management (TBM) model can help. Pinter noted that the latest version of model provides a way for enterprises to work out the cost structure of different AI services and deployment models, bringing together traditional IT financial management (ITFM) and FinOps.

“It’s about being able to look at multiple different disciplines and provide that single pane of glass, where you can get into chargeback and look at SaaS and on-premise applications,” he said. “It’s bringing all that together and providing a vehicle to effectively charge back all the costs that the IT organisation is incurring.”

Ironically, the solution to managing AI costs involves more AI. Pinter expects AI-driven anomaly detection to become essential for preventing bill shocks from misconfigured cloud instances. Natural language chatbots could also replace business intelligence dashboards, allowing executives to query data for instant insights.

But technology alone isn’t enough to drive cost-saving FinOps practices. The single biggest barrier to adopting FinOps, whether in mature cloud markets such as Australia or technology hubs like Taiwan and Singapore, is human resistance.

“It’s the culture shift to get everybody bought into it,” he said. “You might not have executives fully on board, and engineers might be apprehensive. Getting organisational buy-in, where everyone says, ‘Yes, this is what we’re going to do’, is the biggest challenge.”