A single developer has released Colibri, a pure-C inference engine that runs GLM-5.2, a 744-billion-parameter Mixture-of-Experts model, on a consumer machine with roughly 25GB of RAM and no GPU. It does this by keeping only the dense layers resident in memory and streaming the model's routed experts from an NVMe SSD on demand.

What Happened

Colibri shipped as a Show HN project this week under an Apache 2.0 license. The trick is architectural: GLM-5.2 activates only about 40B of its 744B parameters per token, and the engine exploits that. The dense part (attention, shared experts, embeddings, roughly 17B params) stays in RAM at int4 quantization, while the 21,504 routed experts sit on disk and are pulled in through an LRU cache as tokens need them.

The whole thing is written in C with zero runtime dependencies, using AVX2 kernels, int4/int8/int2 quantization, and speculative decoding via the model's native multi-token prediction head. The author built and tested it on a 12-core laptop with 25GB of RAM.

Try It: Run a Frontier Open Model With No GPU

If you have an NVMe SSD with room for the roughly 370GB of expert weights, you can clone the repo, download the GLM-5.2 weights, and run inference locally without a cloud API key or a datacenter GPU. That means a fully offline, private setup for prototyping agents or coding assistants on top of a frontier-class open model. Read the GLM-5.2 model notes first so you know what the model is tuned for (long-horizon agentic and coding tasks).

Why It Matters for Creators

Running open weights locally is usually gated by VRAM, which is exactly why most builders reach for a hosted endpoint. Colibri reframes the constraint around disk instead of GPU memory, putting a 744B model within reach of hardware many creators already own. It pairs naturally with the growing GLM tooling ecosystem, including Zhipu's own GLM-5.2 coding agent.

Key Details

Model: GLM-5.2, a 744B Mixture-of-Experts model (~40B active per token).

RAM needed: ~25GB, plus an NVMe SSD for ~370GB of streamed experts.

License: Apache 2.0.

Speed: 0.05-0.1 tokens/second cold, improving as the expert cache warms. This is a proof of concept, not a production server.

What to Do Next

Treat Colibri as an experiment worth running if you care about local, private inference and have fast storage to spare. Clone it, warm the cache on a repeatable prompt, and see whether the disk-streaming approach fits your workflow before you commit any project to it.