llama.cpp shipped version 0.4.0 on September 4, adding initial support for Alibaba's Qwen3.8-Flash-Next, NVIDIA's Nemotron-3-Puzzle-75B-A9B, and video input for local multimodal models. The v0.4.0 release notes also introduce lazy tensor reading, which pulls model weights on demand instead of loading everything into memory up front.
Try It: Run the Newest Open Models on Your Own Box
Check out the v0.4.0 tag, rebuild, and point it at a GGUF build of Qwen3.8-Flash-Next. The flag to reach for first is --lazy-mode, which defers tensor reads so a model that previously refused to start on your RAM budget has a chance of loading. Pair it with the new --n-cpu-ffn option to push feed-forward layers onto the CPU and keep more of the attention stack resident on the GPU.
If you caption, log, or search footage locally, the multimodal work matters more. llama.cpp now accepts video parts through its multimodal helper and adds support for DeepSeek-V4-Flash-Vision-Exp, so a clip can feed a local vision model without a separate frame-extraction pass first.
Why It Matters for Creators
llama.cpp sits underneath a large share of the local AI stack, so its release cadence sets the pace for the desktop tools built on top of it. NVIDIA said at IFA 2026 that recent llama.cpp work delivers up to 1.9x higher throughput on a GeForce RTX 5090, a claim worth testing against your own workload rather than taking at face value.
The practical effect is that the gap between a frontier API and a model running on hardware you already own keeps narrowing. That matters for anyone editing client footage under an NDA, or simply running enough generations that per-token billing stops making sense. Our llama.cpp and LM Studio comparison covers where each one fits a creator workflow.
Key Details
Version: 0.4.0, released September 4, 2026, marked stable rather than prerelease.
New model support: Qwen3.8-Flash-Next (the qwen4exp architecture, with optimization still pending), NVIDIA Nemotron-3-Puzzle-75B-A9B, DSpark for Nemotron 3.5, and nanbeige4.2-3B.
Memory and serving: lazy tensor reading, per-layer expert routing, KV-cell token tracking, an optimized KV cache restore path, a quantizer RAM cap, and per-slot context limits for the server.
Under the hood: ggml moves to 0.23.0 with sparse flash attention for DeepSeek-V4, GLM, and qwen4exp, plus Apple RDMA transport for RPC.
Release track: the project began consistent semantic versioning at v0.2.0 on August 21. The vX.Y.Z tags mark stable builds, while the older b-numbered tags continue as nightly builds cut from most commits to master.
What to Do Next
Build the v0.4.0 tag rather than tracking master if you want a version you can pin and come back to, since that is the whole point of the stable line. Benchmark --lazy-mode against your normal load path before adopting it, because deferring reads trades startup memory for slower first-token latency on some setups. If you run models across more than one machine, the new per-slot context limits pair well with a router like NVIDIA PAIR, and our coverage of the Qwen3.8-Flash-Next open-weights release has the model details.
