Google · TPU Agent Infrastructure

TPU 8tx and TPU 8i Max

Google has added two eighth-generation TPU systems: TPU 8tx, for confidential AI inference on sensitive data, and TPU 8i Max, which pairs two TPU 8i chips for large context windows and multi-agent workloads. Google says TPU 8tx offers up to 2x lower latency than TPU 8t, and that TPU Sync cuts key-value cache transfer latency and time to first token by more than 90%. Alongside the hardware, Google is open-sourcing an XLA Serving Engine, a TPU agent suite (in preview), TPU Sync, and TorchTPU, which runs PyTorch and Hugging Face models on TPUs.