As demand for AI computing continues to grow, leading technology companies are increasingly developing in-house silicon to lessen their reliance on Nvidia. Microsoft has taken a step with the introduction of its Maia 200 AI accelerator, a custom-built chip designed & claimed to deliver competitive performance while improving efficiency.
The Maia 200 is Microsoft’s latest custom chip, purpose-built for AI inference workloads, which involve using trained models to produce predictions or content. Microsoft says the accelerator brings “dramatic” improvements for AI applications and is already deployed in select Azure data centers in the United States.
Fabricated on TSMC’s leading-edge 3-nanometer node, the Maia 200 is a high-density accelerator featuring over 140 billion transistors within a 750-watt TDP. The design integrates native FP8 and FP4 tensor cores, 216 GB of HBM3e memory, and a substantial 272 MB on-chip SRAM cache to support AI workloads.
Microsoft claims these components combine to make the Maia 200 the highest-performing custom silicon design currently in use by any hyperscale cloud provider. The company released specific performance comparisons, stating the chip is up to three times more powerful than Amazon’s third-generation Trainium accelerator at FP4 (4-bit) precision. It also reportedly surpasses Google’s seventh-generation TPU at FP8 (8-bit) precision. Internally, Microsoft states the Maia 200 offers a 30 percent better performance-per-dollar ratio than its predecessor, the Maia 100.
Performance is rated at over 10 petaFLOPS at FP4 and over five petaFLOPS at FP8, enabling it to run today’s most powerful AI models. Microsoft designed the system-on-a-chip (SoC) to support even larger future models.
One of the big challenges Microsoft wanted to solve was data movement. The Maia 200 tackles that with a new Ethernet-based network design, complete with a custom transport layer and a built-in NIC, giving each chip 2.8 TB/s of two-way bandwidth for AI workloads.
Right now, Maia 200 is running in Microsoft’s Iowa data centers, with more regions set to come online later. Beyond powering Azure workloads, Microsoft is also using the chip to create “synthetic” data, a response to growing worries that there won’t be enough human-generated content to train future AI models.
Microsoft plans to release an official Maia 200 software development kit to support developers and AI startups. The SDK will include a compiler, PyTorch integration, low-level programming tools, and a Maia simulator.
Maybe you would like other interesting articles?