JustVugg · Inference Engine

Colibrì

Apache-2.0 zero-dependency C runtime that runs the 744B-parameter MoE model GLM-5.2 on ~25GB-RAM consumer machines by streaming expert weights from disk across a VRAM/RAM/NVMe tier hierarchy, with MLA attention, speculative decoding, and an OpenAI-compatible API. ~0.05–0.1 tok/s on a 25GB device, up to ~6.8 tok/s with six GPUs.