Agent Atlas

Atlas · frontier

Chips and data centers

Models run on accelerators with fast memory, packed into buildings measured in megawatts.

Established practiceDepth 3 of 4Reviewed 2026-11-12

See it working: Plate V

In plain words

The model you talk to runs on chips built for multiplying large tables of numbers, each with a pool of very fast memory.

How it works

Memory capacity limits how many weights fit on a chip, and memory bandwidth is one factor in how fast it runs. NVIDIA lists the H100 SXM at 80GB and 3.35TB/s, and the B200 at 180GB and up to 8TB/s. Google lists its TPU 8t at 216 GB and 6,528 GB/s. For comparison, Apple lists the M5 Pro Mac mini at 307GB/s.

Going deeper

Epoch AI estimates that the record IT power capacity of a single AI data center has doubled every ten months since mid-2024, reaching about 950 MW at xAI’s Colossus 2. IT power leaves out cooling, and these are Epoch’s estimates, not the companies’. The memory calculator on Plate V puts these chips beside a Mac.

Sources

  1. NVIDIA H100 Tensor Core GPU NVIDIA
    • 80GB
    • 3.35TB/s
    • 3,958 teraFLOPS
    • With sparsity
    • Up to 700W (configurable)
  2. Components — NVIDIA HGX AI Factory (HGX H100, H200, B200) NVIDIA
    • 180GB HBM3e
    • Up to 8TB/s
    • based on eight B200 GPUs, that delivers 144 petaFLOPs of AI performance.
  3. Inside the eighth-generation TPU: An architecture deep dive Google Cloud
    • 216 GB
    • 12.6
    • 6,528 GB/s
  4. The record for power capacity in a single data center has doubled every 10 months Epoch AI
    • Since mid-2024, the record for the largest AI data center by IT power capacity has doubled every ten months. The current record-holder is xAI’s Colossus 2, at an estimated 950 MW.
    • IT power is the power capacity available to chips, servers, and other IT equipment, excluding cooling and other overhead.
  5. Mac mini (M6 or M5 Pro) - Tech Specs Apple
    • 307GB/s memory bandwidth
    • 24GB unified memory
    • 48GB or 64GB

Quotes were checked against the live pages on 2026-11-14.