How is an AI data centre different from a traditional data centre?

An AI data centre is less a different kind of building than a different kind of load. Facilities built for accelerated computing, where servers packed with GPUs or similar accelerators work together, typically concentrate far more power and heat into each rack, depend much more on the network between servers, and run close to full load for long periods. Those three differences ripple through electrical distribution, cooling, layout and operations.

There is no single AI configuration, though. A rack of inference servers answering questions for a chatbot can look much like conventional IT, while a cluster training a large model can be unlike anything the existing room was designed for. The decision has to rest on the actual equipment and workload, whatever the label says.

Where the differences show

Density is the headline

The US Department of Energy's 2024 data centre design guide notes that high-performance computing facilities saw rack densities of 60 kW per compute rack in 2013, with recent figures surpassing 125 kW. It also puts average server utilisation in enterprise settings at generally 20–40%. AI training clusters push in the opposite direction on both counts: dense and busy.

The figure for any particular rack has to come from the documented configuration of the equipment being bought, not from a rule of thumb, and the same goes for the cooling method. ASHRAE's data centre committee, TC 9.9, observed in a 2021 white paper that processor chips available only with liquid cooling were already on the market, with more on the way.

What matters most is where the demand lands. A facility can have spare capacity in total and still be unable to deliver it to particular racks, so power distribution, cooling delivery and floor loading have to be checked at the rack positions you actually intend to use.

The network becomes part of the building

Distributed training depends on fast communication between servers. Published reference designs for large GPU clusters separate the compute, storage and management networks, and their high-speed fabrics set maximum cable lengths. Limits like these constrain where racks can go. Spreading racks out to solve a power or cooling problem can break the network design.

Failure behaves differently

In conventional IT, losing one server usually affects one service. In a training cluster, vendor guidance warns, the failure of a single system in a multi-node job can stop the job on all of them. That shifts attention to power-supply arrangements, cooling continuity and checkpointing in the software, and it means the facility's resilience requirement depends on whether work can be restarted from a saved state.

Four things to have in hand

Four things should be on the table before design begins. The supplier should provide a documented equipment configuration with power, cooling and network requirements per rack. There should be a growth scenario saying how many racks, of what kind, and when. The workload's resilience requirement needs stating, including whether jobs can resume after an interruption. And there should be a monitoring plan that links IT performance to facility conditions, so that a cooling constraint that is slowing the cluster shows up.