Generative AI and data centres: How it works

Each year, data centres around the world power the technologies we use in everyday life. Everything from cloud services and new tools to advancements in existing tech relies on the low- latency connectivity and computing power that modern data centres bring.

Generative artificial intelligence (AI) is no different. In recent years, tools like OpenAI’s ChatGPT have brought generative AI into public awareness. With innovative conversational responses to human prompts, these systems allow us to learn, process and create in entirely novel ways. But this creativity comes with challenges. The most obvious is that they require a phenomenal amount of processing power.

That’s why the computing infrastructure that runs generative AI models is often housed in data centre facilities that can handle these significant higher processing demands. Whilst data centres have been used for artificial intelligence and machine learning for years, generative AI brings new computational challenges in terms of volume, speed, and sustainability.

To highlight the issue, a BBC report showed that generating content can use 33 times the energy of conventional software. This means that data centres hosting generative AI models workloads need to maximise their resources at every turn — balancing quality output with efficiency.

In this blog, we’ll explore how this process works, and the infrastructure required to handle these computing demands, along with the prevalent challenges faced by data centre operators and the ongoing innovations to solve them.

How does generative AI work?

Generative AI — often shortened to Gen AI — works by training on extensive datasets of various materials. It then uses advanced models to produce new outputs based on its learnings.

However, training generative AI models is an intensive process. It uses sophisticated systems capable of handling billions, or even trillions, of parameters. Such extensive training builds the model's ability to generate realistic and contextually relevant responses. And, once trained, the models perform inference, responding to user inputs with new outputs.

Due to the massive datasets and processing involved, creating generative AI systems is heavily dependent on specialised hardware, including:

Central Processing Units (CPUs)

Graphics Processing Units (GPUs)

High bandwidth networking and other powerful hyperscale data centre components.

Combined, these processors can manage the high volume of intensive calculations needed. The systems are often hosted in hybrid cloud environments, providing scalable access to the necessary resources.

However, this infrastructure comes with significant demands, including high energy usage, advanced cooling systems, and complex infrastructure architecture. Thanks to modern data centres, we can train and run such large models — even as datasets grow larger and more specialised.

Generative AI infrastructure: Data centres in demand

As demonstrated, processing the enormous datasets and training generative AI models is a task that deposits exceptional demands on infrastructure. For most organisations, managing this internally is impractical and expensive — it takes extensive hybrid cloud and data centre infrastructure.

Thankfully, data centres can provide such a setup. And this demand is only set to increase. A Goldman Sachs report predicts that data centre energy demand could grow by 160% by 2030 — driven in large part by the widespread adoption of generative AI technologies.

However, the relationship between AI and data centres is increasingly a two-way street. Data centres are currently using, and will increasingly use in future, AI systems to improve key metrics like energy efficiency.

How do data centres support generative AI?

As well as providing the raw computational power, data centres’ networking and storage systems play a vital role in modern AI. Fundamentally, generative AI depends on rapid data transfer to manage large datasets efficiently. To meet this, data centres are investing in scalable storage solutions and ultra-high-speed connectivity.

Many data centre facilities are also incorporating renewable energy sources and improving energy efficiency through highly optimised cooling systems. These changes help manage the environmental impact of generative AI while meeting its infrastructure requirements.

Generative AI in action

Generative AI is already transforming industries across the world. Hundreds of millions of users have tried applications like ChatGPT and DALL·E:

ChatGPT is a large language model (LLM) that produces coherent and logical text responses. It has a wide range of applications but is particularly well-suited for customer service in business given its ability to understand and respond to complex queries, making it a useful tool for enhancing customer interactions and improving service efficiency.

DALL·E, on the other hand, is an advanced AI model that generates detailed and intricate images based on textual descriptions. This capability makes it an invaluable tool for a variety of applications, but it is especially beneficial in ideation and creative industries.

Different types of generative AI models are tailored to specific purposes. Generative Adversarial Networks (GANs) are commonly used to create images and videos. They rely on two networks working together, with one generating content and the other refining it.

Variational Autoencoders (VAEs) take another approach. They produce structured outputs by compressing and then recreating data. Many other types of generative AI models use variations on these primary methods.

H3> Training and inference

As a general rule, generative AI involves two fundamental phases:

Training

Inference

Training builds the model's capability and accuracy using large datasets. This stage requires enormous energy consumption, computational power and time to complete.

Inference involves generating outputs in real-time based on user inputs. There is a lower power demand and energy consumption during this phase, but it still requires robust infrastructure to be in place.

Data centres have a critical role in supporting both the training and inference phases. With advanced configurations for high performance and resource allocation, they help generative AI applications operate at their best while minimising emissions from the increased demand. As a result, enabling generative AI providers to deliver large-scale models to their growing end-user bases.

Navigating the challenges

The growing use of generative AI brings pressing challenges for data centre operators.

One of the biggest concerns is the energy required to train and deploy AI models. For example, projections indicate that energy use originating from AI workloads could exceed the total 2022 data centre consumption within just a few years.

Managing heat generated by high-density, next-generation hardware is another key issue. The compact configurations of processors used in AI systems produce enormous amounts of heat, which requires advanced cooling solutions. Liquid cooling systems are a promising solution but add complexity and cost to data centre operations.

Latency is also a concern. Generative AI models are computationally intensive. They may typically have slower response times compared to traditional systems. As such, improving network infrastructure, including faster data transfer systems, is critical for continuing to deliver results quickly.

Likewise, sustainability is a priority for data centres — particularly those supporting generative AI. Employing renewable energy sources and more efficient designs is helping to reduce environmental impacts and encourage the most effective power usage possible.

To address these challenges, data centres are constantly introducing innovative strategies and investing in cutting-edge technologies. As generative AI uses more data, power and bandwidth in coming years, these data centre advances will be crucial to delivering high-quality service while meeting environmental responsibilities.

These innovations will not only improve efficiency but also make data centre operations more sustainable. Specifically, Digital Realty has introduced intelligent energy management systems that track and optimise power usage in real time, ensuring resources are used effectively while keeping environmental impact low. With a strong focus on sustainable data centre solutions, Digital Realty is tackling the challenges of rising computing demands while creating a more efficient and responsible future.

The road ahead for generative AI and data centres

Generative AI has had far-reaching impacts on almost every industry — and that influence will only grow as the technology matures. Nevertheless, the success of GenAI depends on the ability of data centres to advance and meet its growing demands.

AI models are expected to become more advanced in the future, requiring even greater computational power, bandwidth and storage capacity than they do today. As a result, data centres must continue to innovate in areas such as energy management, infrastructure design, and cooling systems, with renewable energy and energy-efficient practices becoming standard.

To put it simply, data centres are at the heart of generative AI’s future success. As this transformative technology continues to improve, data centres will play an increasingly important role in supporting its development.

Learn more about the data centre of the future.

Internal Links

https://www.digitalrealty.co.uk/resources/solution-briefs/cloud

https://www.digitalrealty.co.uk/expertise/solutions/cloud

https://www.digitalrealty.co.uk/resources/articles/sustainable-data-centre-ai

https://www.digitalrealty.co.uk/data-centers

https://www.digitalrealty.co.uk/resources/articles/data-center-future

External Links

https://www.bbc.co.uk/news/articles/cj5ll89dy2mo

https://www.goldmansachs.com/insights/articles/AI-poised-to-drive-160-increase-in-power-demand

https://www.morganstanley.com/ideas/ai-energy-demand-infrastructure

PDx® Hybrid AI Use Case Featuring Oracle Solutions – AI Private Infrastructure