How to improve AI efficiency beyond cost optimisation

Hidden costs of inefficient AI practices

The rush to adopt AI regardless of strategic fit can often lead to costly missteps.

Inefficient AI operations drive up expenses and reduce operational resilience, ultimately undermining investor confidence. As a result, stakeholders are increasingly focused on managing both the cost and value of AI use cases.

Over-engineering AI by building systems that are more complex than necessary or misaligned with business needs, drives excessive resource consumption. This not only increases operational costs but also exacerbates environmental challenges at a time when sustainability is under growing scrutiny from regulators and stakeholders.

Executing AI efficiency

Educate consumers and builders of AI capabilities on the business value of optimising AI efficiency and keep them informed to motivate their active participation in optimisation practices. For example, highlight how streamlined AI operations strengthen operational resilience, enabling organisations to better withstand disruptions such as electricity supply constraints.

A foundational step towards AI efficiency is disciplined use case prioritisation. Evaluate each opportunity for merit, feasibility and return on investment (ROI) to ensure resources are focused on initiatives most likely to deliver meaningful value and align with organisational capabilities.

Once high impact use cases are identified, the next priority is embedding best practices throughout the AI technology stack and operational processes.

From the tooling perspective, efficiency can be enhanced by moving beyond LLMs alone and leveraging a combination of different AI techniques. This approach enables greater cost-effectiveness, responsiveness, and robustness in enterprise AI systems.

Engineering teams can also optimise apps for AI inference activities, which typically accounts for more energy consumption than training. This can be achieved by using smaller models, inference-optimised hardware, caching intermediate outputs and distributing access points closer to consumption.

On the infrastructure side, AI efficiency goals can be further advanced by partnering with cloud providers that demonstrate verifiable commitments to renewable energy use, high energy and water efficiency standards (such as power or water usage effectiveness) and transparent environmental reporting.

Another critical area for advancing AI efficiency is monitoring and operations. Actively managing and optimising AI workloads, resource allocation and operational processes can yield significant cost and sustainability benefits.

Scheduling large-scale, energy-intensive processes during off-peak hours allows organisations to take advantage of less expensive, greener energy sources. Carefully distinguishing between workflows that require real-time processing versus those suitable for batch processing can also reduce costs.

Additionally, adopting GreenOps extends FinOps principles by applying sustainability metrics across IT operations. Continuously tracking energy consumption, greenhouse gas emissions and waste generation helps ensure ongoing optimisation of AI workloads while supporting broader environmental goals.

These are just a few ways to embed AI efficiency into an organisation. True impact requires action across the overall AI technology stack, including applications, selecting the right AI techniques and tools, and implementing robust data management practices.

Measuring success

AI efficiency metrics should be prioritised based on their impact on business outcomes. These measures reflect how ready a technology stack is to support critical business processes and deliver results.

Start by tracking the percentage of stakeholders trained on efficient AI principles. This metric shows how effectively the organisation is building a culture of efficiency, ensuring employees and leaders understand both the business value and practical steps required for optimisation.

Next, monitor the proportion of AI workloads running on sustainability-aware and reliable infrastructure. Prioritise deployments powered by verified renewable energy sources, measured against provider sustainability benchmarks.

Track data management efficiency by measuring the reduction in total data storage requirements and the volume of data transferred for AI workloads, which indicates improved data handling and lower resource consumption.

Finally, measure asset reuse rates across the portfolio. Evaluate how often models or software components are leveraged in multiple projects to minimise duplication of effort and maximise resource use.

By moving beyond cost optimisation and embedding efficiency across every layer of AI deployments, organisations can unlock sustainable growth while meeting today’s business challenges.

Gabriele Rigon is a director-analyst at Gartner, specialising in natural language technologies and contributes to research relating to AI efficiency, GenAI, AI agents and AI governance.