Together AI · Agent-driven GPU compiler

Together Compiler

Together AI describes Together Compiler, a framework that uses AI agents to rewrite and fuse the GPU kernels in a model's forward pass. The post reports it beats its own hand-written megakernels by 28% on B200 (2,118 vs 1,649 tokens/s on Llama 3.2 1B). It also says CUDA Graphs and PDL on default Torch Inductor kernels recover 98% of megakernel performance, and that multi-kernels with agent loops reach over 1,100 TPS for Llama 1B on H100.

Open the source ↗

Similar Developer Tools releases

More from Together AI

Published by Quanta AI Labs, an AI agency in India. Browse every week in AI News Weekly.