Google announced TPU 8t and TPU 8i at the Cloud Next conference. TPU 8t is specifically designed for training cutting-edge models; TPU 8i is tailored for inference, low-latency AI agent workloads, and more complex inference tasks.

Compared to the previous generation, the performance per watt of the 8t increased by 124%, and the 8i by 117%. The cost-performance ratio of the 8i is also 80% higher than Ironwood.

TPU 8t adds a new layer of high-radix non-blocking switching fabric, the Virgo Network, on top of the traditional 3D Torus, with 47 Pb/s high-radix switching. It can scale to over 1 million TPU chips in a single training cluster via JAX+Pathways. DCN also grew by 4x. The TPU 8i Boardfly pod top layer enforces a fully interconnected OCS across 36 groups. The usage of long-range optical modules has been significantly revised upwards (possibly 2× or more), and the speed has been upgraded from 800G to 1.6T (3.2T in some scenarios), with the ASP per port also rising accordingly. Google's new TPU clusters use approximately 6.8 times more HBM than the previous generation.

It's getting cheaper as it rises.