Tencent (Hunyuan) · MoE Multimodal LLM

Hy4-flash

Hy4-flash is a mixture-of-experts model with 220B total parameters (including 64B Engram parameters) and 12B active per token, accepting text and image inputs with a 1M-token context window. It adds hybrid sparse and sliding-window attention, LatentMoE routing and n-gram embeddings, plus a built-in 0.85B speculative drafter (DFly2). The page compares its base model with Hy4 Base and DeepSeek-V4-flash-Base on knowledge, reasoning, math, code and multimodal benchmarks. It is released under the Tencent Hy licence on Hugging Face, ModelScope and GitCode.

Open the source ↗

Similar Language Models releases

Published by Quanta AI Labs, an AI agency in India. Browse every week in AI News Weekly.