In the present day, we’re unveiling the following Fairwater web site of Azure AI datacenters in Atlanta, Georgia. This purpose-built datacenter is linked to our first Fairwater web site in Wisconsin, prior generations of AI supercomputers and the broader Azure world datacenter footprint to create the world’s first planet-scale AI superfactory. By packing computing energy extra densely than ever earlier than, every Fairwater web site is constructed to effectively meet unprecedented demand for AI compute, push the frontiers of mannequin intelligence and empower each particular person and group on the planet to realize extra.
To fulfill this demand, now we have reinvented how we design AI datacenters and the programs we run inside them. Fairwater is a departure from the normal cloud datacenter mannequin and makes use of a single flat community that may combine tons of of 1000’s of the newest NVIDIA GB200 and GB300 GPUs into an enormous supercomputer. These improvements are a product of many years of expertise designing datacenters and networks, in addition to learnings from supporting among the largest AI coaching jobs on the planet.
Whereas the Fairwater datacenter design is effectively suited to coaching the following era of frontier fashions, it’s also constructed with fungibility in thoughts. Coaching has advanced from a single monolithic job into a spread of workloads with totally different necessities (resembling pre-training, fine-tuning, reinforcement studying and artificial knowledge era). Microsoft has deployed a devoted AI WAN spine to combine every Fairwater web site right into a broader elastic system that permits dynamic allocation of various AI workloads and maximizes GPU utilization of the mixed system.
Under, we stroll by means of among the thrilling technical improvements that help Fairwater, from the way in which we construct datacenters to the networking inside and throughout the websites.
Most density of compute
Fashionable AI infrastructure is more and more constrained by the legal guidelines of physics. The velocity of sunshine is now a key bottleneck in our potential to tightly combine accelerators, compute and storage with performant latency. Fairwater is designed to maximise the density of compute to attenuate latency inside and throughout racks and maximize system efficiency.
One of many key levers for driving density is bettering cooling at scale. AI servers within the Fairwater datacenters are linked to a facility-wide cooling system designed for longevity, with a closed-loop method that reuses the liquid repeatedly after the preliminary fill with no evaporation. The water used within the preliminary fill is equal to what 20 properties eat in a 12 months and is barely changed if water chemistry signifies it’s wanted (it’s designed for 6-plus years), making it extraordinarily environment friendly and sustainable.
Liquid-based cooling additionally offers a lot larger warmth switch, enabling us to maximise rack and row-level energy (~140kW per rack, 1,360 kW per row) to pack compute as densely as doable contained in the datacenter. State-of-the-art cooling additionally helps us maximize utilization of this dense compute in steady-state operations, enabling massive coaching jobs to run performantly at excessive scale. After biking by means of a system of chilly plate paths throughout the GPU fleet, warmth is dissipated by one of many largest chiller crops on the planet.
One other method we’re driving compute density is with a two-story datacenter constructing design. Many AI workloads are very delicate to latency, which implies cable run lengths can meaningfully impression cluster efficiency. Each GPU in Fairwater is linked to each different GPU, so the two-story datacenter constructing method permits for placement of racks in three dimensions to attenuate cable lengths, which in flip improves latency, bandwidth, reliability and value.
Excessive-availability, low-cost energy
We’re pushing the envelope in serving this compute with cost-efficient, dependable energy. The Atlanta web site was chosen with resilient utility energy in thoughts and is able to reaching 4×9 availability at 3×9 price. By securing extremely accessible grid energy, we are able to additionally forgo conventional resiliency approaches for the GPU fleet (resembling on-site era, UPS programs and dual-corded distribution), driving price financial savings for patrons and quicker time-to-market for Microsoft.
We’ve got additionally labored with our business companions to codevelop power-management options to mitigate energy oscillations created by massive scale jobs, a rising problem in sustaining grid stability as AI demand scales. This features a software-driven resolution that introduces supplementary workloads in periods of lowered exercise, a hardware-driven resolution the place the GPUs implement their very own energy thresholds and an on-site vitality storage resolution to additional masks energy fluctuations with out using extra energy.
Chopping-edge accelerators and networking programs
Fairwater’s world-class datacenter design is powered by purpose-built servers, cutting-edge AI accelerators and novel networking programs. Every Fairwater datacenter runs a single, coherent cluster of interconnected NVIDIA Blackwell GPUs, with a sophisticated community structure that may scale reliably past conventional Clos community limits with current-gen switches (tons of of 1000’s of GPUs on a single flat community). This required innovation throughout scale-up networking, scale-out networking and networking protocol.
By way of scale-up, every rack of AI accelerators homes as much as 72 NVIDIA Blackwell GPUs, linked by way of NVLink for ultra-low-latency communication throughout the rack. Blackwell accelerators present the very best compute density accessible right now, with help for low-precision quantity codecs like FP4 to extend whole FLOPS and allow environment friendly reminiscence use. Every rack offers 1.8 TB of GPU-to-GPU bandwidth, with over 14 TB of pooled reminiscence accessible to every GPU.
These racks then use scale-out networking to create pods and clusters that allow all GPUs to operate as a single supercomputer with minimal hop counts. We obtain this with a two-tier, ethernet-based backend community that helps huge cluster sizes with 800 Gbps GPU-to-GPU connectivity. Counting on a broad ethernet ecosystem and SONiC (Software program for Open Community within the Cloud – which is our personal working system for our community switches) additionally helps us keep away from vendor lock-in and handle price, as we are able to use commodity {hardware} as a substitute of proprietary options.
Enhancements throughout packet trimming, packet spray and high-frequency telemetry are core elements of our optimized AI community. We’re additionally working to allow deeper management and optimization of community routes. Collectively, these applied sciences ship superior congestion management, fast detection and retransmission and agile load balancing, guaranteeing ultra-reliable, low-latency efficiency for contemporary AI workloads.
Planet scale
Even with these improvements, compute calls for for big coaching jobs (now measured in trillions of parameters) are rapidly outpacing the facility and house constraints of a single facility. To serve these wants, now we have constructed a devoted AI WAN optical community to increase Fairwater’s scale-up and scale-out networks. Leveraging our scale and many years of hyperscale experience, we delivered over 120,000 new fiber miles throughout the US final 12 months — increasing AI community attain and reliability nationwide.
With this high-performance, high-resiliency spine, we are able to immediately join totally different generations of supercomputers into an AI superfactory that exceeds the capabilities of a single web site throughout geographically various places. This empowers AI builders to faucet our broader community of Azure AI datacenters, segmenting visitors primarily based on their wants throughout scale-up and scale-out networks inside a web site, in addition to throughout websites by way of the continent spanning AI WAN.
This can be a significant departure from the previous, the place all visitors needed to journey the scale-out community whatever the necessities of the workload. Not solely does it present clients with fit-for-purpose networking at a extra granular degree, it additionally helps create fungibility to maximise the flexibleness and utilization of our infrastructure.
Placing all of it collectively
The brand new Fairwater web site in Atlanta represents the following leap within the Azure AI infrastructure and displays our expertise operating the biggest AI coaching jobs on the planet. It combines breakthrough improvements in compute density, sustainability and networking programs to effectively serve the large demand for computational energy we’re seeing. It additionally integrates deeply with different AI datacenters and the broader Azure platform to kind the world’s first AI superfactory. Collectively, these improvements present a versatile, fit-for-purpose infrastructure that may serve the complete spectrum of contemporary AI workloads and empower each particular person and group on the planet to realize extra. For our clients, this implies simpler integration of AI into each workflow and the flexibility to create progressive AI options that have been beforehand unattainable.
Discover out extra about how Microsoft Azure may help you combine AI to streamline and strengthen growth lifecycles right here.
Scott Guthrie is answerable for hyperscale cloud computing options and companies together with Azure, Microsoft’s cloud computing platform, generative AI options, knowledge platforms and data and cybersecurity. These platforms and companies assist organizations worldwide remedy pressing challenges and drive long-term transformation.
Editor’s word: An replace was made to extra clearly clarify how we optimize our community.