Nanbeige LLM Lab · Small Language Models

Nanbeige4.2-3B

4B-total / 3B-non-embedding agentic model under Apache 2.0, with a 262,144-token context and a Looped Transformer architecture (LoopSplit, mHC depth attention). Reports 63.6% on SWE-Bench Verified and 87.4% on GPQA-Diamond, above Qwen3.5-9B's 53.1 at a third the size. Trained on 28T tokens; runs on vLLM, SGLang, llama.cpp and Ollama.