Green coding - smartR AI: Cleaning up AI training & inference
This is a guest post for the Computer Weekly Developer Network written by Oliver King-Smith, founder and CEO of smartR AI.
The company insists that it is committed to developing life-changing Artificial Intelligence (AI) applications based on the evolution of interactions, behaviour changes and emotion detection.
Focusing on behavioural intelligence and interconnections with the Internet of Things, smartR AI uses AI applications to understand, interpret, predict and respond to complex scenarios as intelligence moves to the edge of the network. The company says its engines ensure optimal efficiency and performance, improve quality and reduce human error using AI and machine learning, deep learning, reinforcement learning and more.
King-Smith writes in full as follows…
There are growing concerns about the rapid expansion of Artificial Intelligence and the associated power and water supply needs that come with it. The new Nvidia DGX computers are the gold standard for AI work, consuming over 10 kW of power. Big tech companies will buy millions of these systems this year, using more electricity than the entire city of New York.
However, it’s not just the electric power required to run these computers that is of concern.
They generate immense amounts of heat and require a lot of cooling, often using twice the power than the actual computer, meaning this 10kW machine actually uses 30kW when running. These new servers could use three times as much electricity as the entire state of California did in 2022! To alleviate the situation, server farms began using large amounts of water to cool their equipment, saving power but depleting precious freshwater resources.
AI training & inference
There are two important steps in AI: training and inference.
Training creates the models and needs to be done once for each new model. For example, when developing and training new models smartR AI has utilised the super computer at EPCC, Edinburgh University and it can use several months of compute time to finish up an advanced multimodal model for interpreting images and text. Inference, where the models are run to generate results, might occur millions or even billions of times.
While inference is less compute-intensive than training, the sheer number of times models are inferred means significant energy is used in both stages. With the Wall Street Journal estimating big tech is buying $50 billion worth of Nvidia chips this year, which translates to millions of systems it’s clear that a lot of inferencing will take place.
Companies could commit to using only renewable power for training, locating their server centres in areas with excess renewable energy, like Scotland. If the power supply is insufficient for a period of time, they could shut down servers to reduce their environmental impact, albeit at the cost of slowing development. This green energy strategy would significantly reduce the footprint of training new models.
Inference, however, is trickier. Time-sensitive applications, like AI-assisted medical diagnosis, cannot wait for renewable power availability.
Cleaner edge
High-performance AI chips being developed by companies like ARM and Nvidia, along with startups, will help move processing from server centres to the edge, where cooling and water aren’t required.
Using more efficient algorithms, such as Mixture of Experts models that utilises up to 70% less energy than comparable models, will also contribute to reducing inference energy usage. Researchers at universities are increasingly focusing on developing greener AI solutions. And since 2020, there has been an explosion of interest in scientific publications exploring methods to create more environmentally friendly algorithms.
With AI is on a collision course with unsustainable energy usage and threatening to undermine its potential to change the world for the better, it is imperative that the AI industry prioritises energy efficiency and sustainability measures.
Without a concerted effort to address this issue, the transformative power of AI could come at an unacceptable cost to our environment and future generations.
