Gartner: Prioritise governance to beat AI hype

Data and IT leaders are under pressure to deliver business outcomes from artificial intelligence (AI) initiatives amid ongoing industry hype and fears of a bursting bubble, but achieving true business value goes beyond return on investment (ROI).

That’s because AI represents more than just a technology shift, Gartner’s vice-president analyst Jorg Heizenberg noted at the research firm’s Data and Analytics Summit keynote in Sydney last week, adding that it marks a change that could be just as profound as the arrival of the internet.

In navigating the AI era, Gartner director-analyst Georgia O’Callaghan noted that while nearly three in five organisations had put an AI solution in production in 2025 and four in five are doubling down on AI today, “you can’t just continue to increase your investments in AI without getting clarity on the goals and ambition of your organisation”.

Heizenberg warned that data and analytics professionals should redefine their AI ambitions with input from stakeholders, particularly regarding their tolerance for AI disruption. Those with a low tolerance can choose a cautious approach, carefully assessing risk and following the safest course. Those with a greater appetite for disruption can take a more opportunistic approach, while organisations with a high tolerance might dare to be pioneers, even if that means taking the biggest risks.

However, one of the first questions stakeholders ask is: “What is this going to cost?” This is a difficult question to answer because AI costs are highly unpredictable and often hidden, O’Callaghan pointed out. The problem is compounded by vendors using pricing models based on metrics that are difficult to forecast, such as graphics processing unit (GPU) hours and token consumption.

Gartner’s research shows that while six out of 10 IT leaders are worried about AI agents running up unexpected costs, only two out of 10 data and AI leaders are concerned that unpredictable pricing might limit the value they get from the technology. This disconnect should be a wake-up call, Heizenberg said.

“AI can be an expensive lesson,” warned O’Callaghan, noting that less than half of organisations manage and optimise their AI-related spending. Organisations should track expenses from the outset – especially during prototyping – and adopt cost-driven design to understand the financial impact of various components before going into production. For instance, teams should explore the cost implications of using different large language models (LLMs), or even small language models (SLMs), to power an AI agent.

When communicating with stakeholders, however, the focus should remain on value rather than just cost – and there is more to value than money. Heizenberg highlighted North Yorkshire Council, which created a digital citizen named Dotty and mapped her journey through public services to make the impact of data relatable for all employees.

For example, transposing two digits in a home address might result in a tradesperson being sent to the wrong house to install a handrail for an elderly person. That wasted journey carries a direct financial cost, but there are also ripple effects: what if the lack of a handrail results in the resident falling and suffering a serious injury?

The change management and training effort for AI tools takes nearly twice as long as implementing the AI solution itself, which means planning for longer timelines and higher costs than for any other technology implementation you have managed before

Whatever an organisation’s ambition, foundational investments are key. A 2025 Gartner survey on modern data realisation found that respondents who were most satisfied with the outcomes of their AI use cases spent 30% more on foundational activities, such as data management, governance and talent, compared to those who were unsatisfied.

Other Gartner surveys found 59% of IT leaders felt they were being pushed into adopting generative AI (GenAI) tools before they were ready, while 61% felt pressure from senior leaders, directors, or stakeholders to move forward with AI.

One of the biggest issues is whether an organisation’s data is secure and well-governed enough to be opened up to further AI applications, including autonomous agents.

“We need to prevent the exposure of the wrong data to the wrong people, applications or LLMs with AI governance, and avoid inaccuracies, misunderstandings, and hallucinations with a well-designed context layer,” said O’Callaghan. “This will help to ensure that your data is AI-ready, trusted, and aligned to the use case.”

She added that this highlights the importance of repositioning governance as a business value accelerator, rather than a function focused purely on compliance.

To improve AI governance, the analysts suggested three key steps. First, organisations should connect their existing governance groups, such as risk, data and cyber security, into a unified AI governance team. Gartner predicts organisations connecting governance bodies in this way will experience a 10% greater business impact than those that do not.

The second step is to rationalise governance. This involves having the unified team review and consolidate various policies into a clear, consistent framework that reflects the organisation’s risk tolerance and cultural values around responsible AI use.

Finally, governance must be embedded across both the culture and the technology of the business. This requires shifting the organisational mindset from compliance to one where everyone understands how to use data responsibly and ethically. Technologically, leaders should adopt policy-as-code so rules are automatically enforced throughout the tech stack. Gartner predicts that by 2028, organisations using specialised governance tools will decrease the cost of regulatory compliance by up to 20%.

Even when data is well-governed, context remains critical. If an employee asks how many active customers the business has, the answer depends on the definition of ‘active’. Does it mean someone who made a recent purchase, holds an ongoing subscription, or recently visited the website?

In the absence of context, an LLM can easily misunderstand the prompt and rapidly amplify that error.

“It’s time to build an integrated context realisation layer – a layer that connects every piece of information so everyone and everything, people and agents alike, can see the bigger picture and make more informed decisions,” said Heizenberg.

While semantic layers are becoming commonplace, they are no longer sufficient on their own. Organisations are now experimenting with ontologies, knowledge graphs, and other methods to attach deeper meaning to data. Combining these approaches yields far more accurate results.

Another challenge is that technology is evolving faster than the workforce can adopt it. “If you're investing in AI without investing in your people, you are throwing money away,” warned O’Callaghan. “The change management and training effort for AI tools takes nearly twice as long as implementing the AI solution itself, which means planning for longer timelines and higher costs than for any other technology implementation you have managed before.”

To counter this, a ‘mindset, skillset, toolset’ approach is highly effective. IT leaders must ask: What mindset obstacles exist in the organisation, and how can they be overcome? What skills gaps are present, and how can they be remedied? Only after addressing mindset and skillset should leaders ask what tooling changes are needed.

Finally, there is the ongoing concern about AI-driven job losses. Gartner found that 34% of CIOs expect to reduce the size of their workforce over the next three years. Conversely, only 4% of chief data officers (CDOs) have decreased their team size in the past year, while 44% have expanded their teams.

“Currently, we’re not seeing much reduction in data and analytics team size, but this is happening in other areas,” said O’Callaghan. She noted, however, that some organisations may be using the introduction of AI as a convenient excuse for layoffs that would have occurred regardless.

“The value of human skills and talent will still sit at the core of delivery teams, but these teams will now combine human expertise with AI agents to make more productive, AI-powered fusion teams,” she concluded.