What Is an AI Data Center?

An AI data center works by pairing dense GPU compute with power, cooling, and networking systems sized for sustained, high-utilization workloads rather than the bursty, mixed traffic a traditional data center was designed around. Every other design decision in the building follows from that one difference.

The compute layer is GPUs, not CPUs. A GPU splits a problem into thousands of smaller calculations it solves in parallel, which is exactly the shape of a neural network's math. Training a large model can pull power utilization north of 80% from every accelerator in a cluster for weeks at a stretch, well above the 50-60% typical of general cloud computing workloads (SemiAnalysis, Datacenter Anatomy Part 1, 2024). That sustained draw is what pushes rack density into triple digits and why the facility's electrical design starts from the chip outward rather than from the building inward.

The networking layer has to move data between thousands of GPUs without becoming the bottleneck the GPUs are waiting on. Inside a single rack, high-bandwidth interconnects such as NVLink move data between GPUs in nanoseconds; across racks, fabrics like InfiniBand or RDMA-capable Ethernet stitch multiple racks into one training cluster. We cover this stack in depth, including the scale-up, scale-out, and scale-across distinction that governs multi-data-center training, in our AIDC networking breakdown.

The cooling layer exists because none of the above works without it. Air cannot pull enough heat off a 130kW rack fast enough to keep the chips inside their thermal envelope, so AI data centers now default to direct-to-chip liquid cooling or, at the extreme end, full immersion. Every watt not spent forcing air through a hot aisle is a watt available for compute, which is why cooling architecture is now a capacity decision as much as a facilities one.

The stack, in order: GPUs and accelerators for compute, high-bandwidth interconnects and network fabric to keep them fed, high-throughput storage so data does not idle the GPUs waiting on it, and liquid cooling plus dedicated power distribution to keep the first three running continuously. Remove any one layer and the other three sit idle. That is the entire design logic of an AI data center compressed into one sentence, and it is why retrofitting an old building rarely works as well as designing one from the electrical system up.

Four kinds of organizations build and operate AI data centers, and the differences between them come down to who owns the infrastructure and who is allowed to use it. Hyperscalers such as Microsoft, Google, and Amazon self-build the majority of their capacity and lease the rest, deploying it internally and to cloud customers. Neoclouds are GPU-specialist operators that build or lease capacity specifically to rent to AI labs and enterprises, without the broader cloud product line a hyperscaler carries. Colocation operators own the building and the power connection and lease rack space to any of the above. Sovereign and enterprise builders construct facilities for a single nation's or company's exclusive use, usually where data residency or security requirements rule out a shared environment entirely.

That last category is a growing share of new capacity for a specific reason: geopolitical exposure. Nations building AI capability without full ownership of the infrastructure it runs on are exposed to export controls, vendor lock-in, and third-party access to sensitive workloads, none of which are acceptable for defense, healthcare, or critical-infrastructure use cases. This is the argument behind sovereign AI data centers as a distinct build category rather than a regional variant of the same hyperscale model.

For enterprises without a sovereignty requirement, the same ownership logic still applies at a smaller scale. A company running AI workloads on infrastructure it does not control is dependent on someone else's capacity, pricing, and roadmap. Enterprise AI data centers exist to give a single organization dedicated compute, storage, and networking it operates on its own terms, without carrying the multi-year construction timeline of a hyperscale build.

Demand keeps growing because model training and inference now consume power at a scale the existing facility pipeline was not built for, and the gap between the two is the central constraint of the entire industry. SemiAnalysis tracks more than 5,000 individual data center facilities worldwide, reconciling accelerator shipments against the electrical capacity actually available to host them, specifically to identify which operators and regions run short of power first (SemiAnalysis, Datacenter Industry Model, 2025). That reconciliation exists because announcement volume and buildable power are no longer the same number.

The scale of a single new AI data center illustrates why. One NVIDIA NVL72 rack draws roughly 120-130kW, and a single gigawatt-scale AI factory requires on the order of 8,300 such cabinets before accounting for cooling and distribution losses, a density that pushes power and land requirements well past what a single traditional facility footprint was designed to hold. Multiply that by the dozens of gigawatt-class campuses under construction or planned across the US, the Gulf states, and Asia, and the constraint stops being GPU supply and becomes grid interconnection, transformer lead times, and water rights, the physical inputs a chip roadmap cannot accelerate.

This is also why the industry is diversifying away from a single deployment model. Hyperscale self-builds remain the largest category by megawatt, but colocation, enterprise-dedicated builds, and sovereign infrastructure are all growing faster than the hyperscale share alone, because no single ownership model can absorb demand growing this quickly on its own.

Every question in this guide eventually reduces to one: who controls the infrastructure a given AI workload runs on. Build it, and you own the timeline, the power contract, and the roadmap; lease or colocate it, and you trade that ownership for speed. Neither answer is wrong, but treating it as a foregone conclusion, defaulting to whichever model a vendor pitches first, is how organizations end up dependent on capacity, pricing, and availability they do not control. Understand the tradeoff before choosing the deployment model, not after.