Does generative AI run on thin air?

It then continues "The training process for large-scale AI models like mine typically requires significant computational power and energy resources. However, the exact energy usage can vary depending on factors such as the hardware infrastructure, training duration, and optimization techniques employed.”

Use is on the rise

As more and more companies are releasing generative AI services such as ChatGPT, stand-alone or integrated into other products, the question arises how costly they are on the environment and in energy use. Are these super large language models or other large AI models trained using renewable energy or coal? How much energy does it take to train the first round? What about the updates and re-training? Then what about the use-phase, the inference, how much energy is used for each request or answer?

Generative artificial intelligence (AI) has captivated our imagination with its ability to create engaging content. From stunning images to intricate music compositions, this technology has seen remarkable advancements. Several companies such as Meta, Google, OpenAI and Microsoft have released services based on this technology and the use has exploded. However, as we delve into the world of generative AI, it is essential to consider the energy use associated with training and running these models.

Energy Use Challenges

Training generative AI models necessitates substantial computational power, often requiring high-performance GPUs or specialized hardware accelerators. This, combined with extended training times spanning days or weeks, results in significant energy use. The training process typically takes place in large data centers designed to optimize efficiency.

In the use phase, the challenge lies in bringing the computation as close as possible to the end user. This necessitates deploying high-performance computers in smaller containers or cabinets at the edge or far-edge nodes, enabled by technologies like 5G or 6G. There, the processors need to be advanced also due to the requirement of low latency for the end user.

The next problem is the limitations of the CMOS technology. It is harder to make the processors more efficient without making them more complex and light-up more transistors, it means less dark silicon. This will result in higher heat flux on the chip surface soon in the level of a nuclear power station. It will also cause the need of lower processor case temperatures 50-60 degrees C. All this will make it harder to cool using air and will require liquid cooling, on-chip or immersion.

In summary, the energy use challenges for advanced AI will happen at two places. It will be experienced in the central hyper-sized data centers for training of models and in the small edge nodes in the use-phase.

Mitigating Energy Use

One approach to address energy use is to optimize the hardware infrastructure used for training and use of generative AI models. This includes developing energy-efficient GPUs and exploring specialized processors designed specifically for AI workloads.

For thermal management of the increased heat flux, efficient cooling systems and power management techniques can also contribute to reducing energy usage as well as making it feasible to operate. The use of holistic cooling and targeted liquid cooling will be needed.

A comprehensive approach to environmental responsibility involves life cycle management, encompassing all stages from hardware production to end-of-life considerations. This includes sustainable material sourcing, operational efficiency, water management, and responsible recycling and reuse practices.

A forgotten piece in the puzzle will be the software optimization. The modern software development paradigm is time-to-market. It’s good for the speed of the model development, but in the use-phase a production-like optimized software stack, from the operating system to the application layer, is needed for efficiency. Maybe even assembly language coding will do a comeback like an old hockey player.

Researchers are constantly working on improving the efficiency of generative AI models. Techniques like model compression, knowledge distillation, and network architecture design aim to achieve comparable performance with reduced computational requirements. By minimizing the model's complexity, the energy use during training and inference phases can be significantly reduced.

Embracing renewable energy sources for powering data centers is a vital step towards offsetting the carbon footprint of generative AI training. Many organizations are transitioning to renewable energy, ensuring a more sustainable approach to AI model development. Additionally, considering the location of data centers and training facilities, favoring regions with abundant renewable energy and cool climate like Sweden, can further enhance the sustainability of generative AI.

Encouraging collaborations among researchers, industry experts, and policymakers can drive innovations that promote energy-efficient practices in generative AI. Sharing best practices, developing standards, and conducting further research on reducing energy use will contribute to a more sustainable future for AI technology.

Final words

Generative AI holds immense potential to revolutionize various industries and fuel creative endeavors. However, the energy challenges associated with training and deploying these models call for responsible practices. By optimizing hardware infrastructure, implementing efficient cooling and power management solutions, prioritizing renewable energy, considering the entire life cycle, optimizing models and software, and fostering collaborative research, we can strike a balance between harnessing the power of generative AI and ensuring a sustainable AI ecosystem for future generations.

And RISE is on the case

We at ICE data center are happy to help you if more questions about AI, data centers and energy use come up—please get in touch if you have any questions!