Together AI · Continual Inference Compiler

Together Compiler (TKCC)

Together AI introduces Together Compiler (TKCC), an agent-driven loop that continually optimises inference engines for changing traffic, hardware and models. It keeps engines persistent and hot-reloads candidate forward passes, cutting serving startup from 467 seconds to 9.0 seconds. The company reports up to 2.6x higher TPS across several open models, 1111 TPS on MiniMax M3.1-Flash and 1205 TPS on DeepSeek v4.1-Flash on the InferenceX low-latency benchmark, and 1.7x over open-source megakernels on Llama 1B at batch size 1. A paper is linked; licence and availability are not stated on the page.