Google · KV Cache Transfer

TPU Sync

Google has released TPU Sync, an Apache-2.0 C++ library with Python wheels for PyTorch and JAX that moves KV caches and model weights across TPU HBM, host DRAM, storage and the data centre network. Google reports over 90% lower KV transfer latency on 235B to 480B MoE models, with Qwen3 235B KV transfer falling from several seconds to about 230 ms in its internal tests on TPU v7x. The README says it is not yet recommended for general use, and the RL and storage tiering features are experimental.

Open the source ↗

Similar Developer Tools releases

More from Google

Published by Quanta AI Labs, an AI agency in India. Browse every week in AI News Weekly.