SJTU, Tencent Hunyuan & Shanghai Innovation Institute · Unified Speech Editing

AuK

A 1.5B foundational model unifying speech generation and editing through natural-language instructions and audio context, combining a multimodal LLM for semantic conditioning, an audio VAE trained jointly on speech, general audio and music for acoustic conditioning, and a hybrid rectified-flow transformer running dual-stream MMDiT blocks into single-stream DiT blocks. Trained on roughly 3.03 billion instruction-audio instances over 1.95 million hours. The distilled AuK-Flash does four-step inference without classifier-free guidance for a claimed 4.5x wall-clock speedup. Code and weights released; no licence or numeric scores on the page.