Institute of Foundation Models (K2 Team) · Open Model Fleet

K2-Horizon

K2-Horizon is a fleet of six fully open language models, from a 0.9B dense model to a 375B-A23B mixture-of-experts model, aimed at reasoning and agentic work. The authors report that the models lead or are competitive with strong open models of their size on agentic and coding benchmarks. The paper introduces Mixture-of-Value Attention (MoVA) and softplus attention gating, and Uno, lossless diffusion adapters that speed up decoding by up to 2.2x without a separate draft model. The release includes intermediate checkpoints, pre-training data, training logs, the xLLM and RL360 frameworks, and roughly 10T synthetic pre-training tokens. Artifacts are available at ifm.ai/k2.

Open the source ↗

Similar Language Models releases

Published by Quanta AI Labs, an AI agency in India. Browse every week in AI News Weekly.