Together AI · Agent-driven GPU compiler
Together Compiler
Together AI describes Together Compiler, a framework that uses AI agents to rewrite and fuse the GPU kernels in a model's forward pass. The post reports it beats its own hand-written megakernels by 28% on B200 (2,118 vs 1,649 tokens/s on Llama 3.2 1B). It also says CUDA Graphs and PDL on default Torch Inductor kernels recover 98% of megakernel performance, and that multi-kernels with agent loops reach over 1,100 TPS for Llama 1B on H100.
Open the source ↗Similar Developer Tools releases
- cuBLAS Ozaki-II and 2M FP64 Algorithms
NVIDIA · FP64 GEMM Emulation · Week 47, 2026
- TPU Sync
Google · KV Cache Transfer · Week 47, 2026
- SGLang-Neuron
RadixArk with Annapurna Labs (AWS) · Trainium inference plugin · Week 47, 2026
- TLX (Triton Low-Level Extensions)
Meta (Engineering at Meta) · GPU Kernel Extension · Week 47, 2026
- NVIDIA OpenShell, Snapshot and Kubernetes Engine
NVIDIA · Agent Runtime Infra · Week 47, 2026
- TPU 8tx and TPU 8i Max
Google · TPU Agent Infrastructure · Week 47, 2026
More from Together AI
- Together Compiler (TKCC)
Together AI · Continual Inference Compiler · Week 46, 2026
Published by Quanta AI Labs, an AI agency in India. Browse every week in AI News Weekly.