Alibaba Qwen · Qwen4 Preview MoE

Qwen3.8-Flash-Next

Open-weight preview of Qwen's next-generation architecture: a multimodal mixture-of-experts with 125B total and 6B active across 512 experts routed top-10 plus one shared over 48 layers, alongside a separate 51B-parameter N-gram embedding table. It combines Gated DeltaNet with Qwen Sparse Attention, adds a gated residual mechanism and the Muon optimiser, and runs a native 262,144-token context extensible to 1M via YaRN. Reported: GPQA Diamond 91.7, SWE-bench Pro 62.5, DeepSWE 1.1 58.7, AndroidWorld 84.5. Released under qwen-community-1.0; the served Qwen3.8-Flash shares the architecture at $0.16 and $0.47 per million input and output tokens.