llama.cpp 0.5.0: Six New Models, Two You Can Run

llama.cpp 0.5.0 shipped on 23 September 2026 at 20:50 UTC, 19 days after 0.4.0, and its release notes list six new model families under a single heading: HRM-Text (DFM Mimir 1B), MiMo-V2.6, HunyuanOCR, Nemotron, Qwen4Exp, and Muse Glimmer. That heading is the most-read part of any llama.cpp release, because for the 129,337 people who have starred the project, "new model support" is the difference between reading about a model and running it.

Six arrivals is the headline. The download and file-size data on Hugging Face tells a narrower story. Two of the six have quantized files that fit on a consumer graphics card today, one entry is preparatory plumbing for a model that has not been released, and the model that dominated open-weights coverage this week has exactly one GGUF on the Hub with zero downloads.

This is not a criticism of the release. It is the gap between "supported" and "runnable," and it is the thing worth knowing before you spend an evening on a download that will not fit.

What 0.5.0 actually shipped

The release describes itself as focused on "backend performance and correctness, broader model coverage, and more robust server/router operation." It bumps the vendored tensor library to ggml v0.25.0, which landed the same day roughly 12 hours earlier and raises the RPC protocol to major version 7.

The three items the maintainers chose as highlights are not models at all. They are CUDA conv2d acceleration via implicit GEMM, Metal MoE and SSM_CONV fusion optimizations, and the ability for the server to bind to multiple addresses. The last one now accepts comma-separated TCP addresses and UNIX sockets through --host.

On the API side, 0.5.0 adds llama_adapter_lora_init_from_file_ptr() for loading a LoRA from an already-open file, a LLAMA_VOCAB_TYPE_TEST dummy tokenizer, and input_image support in server function-call outputs. The predecessor release, covered in our writeup of llama.cpp 0.4.0 and its video input support, set the pattern this one continues: the multimodal surface keeps widening a little with each minor version.

The six new models, ranked by what you can download

Every figure below was read from the Hugging Face API on 23 September 2026. Download counts are a moving snapshot; file sizes are not. Sizes are weights only, so budget additional memory for the KV cache on top of each number.

Muse Glimmer 30B is the release's centre of gravity even though it gets one line in the notes. Its GGUF repository has 711,072 downloads, 342 likes, and has been up since 9 August 2026. The --fuse-qkv flag added in 0.5.0 is a throughput optimization aimed squarely at the single most-downloaded model in the new-models list. We covered the model itself when it launched as an open 30B on-device agent model.

The Q4_K_M file is 16.76 GB and the multimodal projector adds 1.4 GB, for 18.16 GB of weights. That clears a 24 GB card with 5.84 GB left for context. The dynamic Q4_K_XL variant at 19.65 GB is tighter and worth skipping unless you have measured a quality difference on your own prompts.

HunyuanOCR is the quiet winner for anyone doing document work. The base model has 713,348 downloads and has been on the Hub since 18 November 2025, so this is not a new arrival; 0.5.0 adds DFlash acceleration to a model that already had GGUF support since April. At 0.58 GB for the Q8_0 weights, it runs on essentially anything.

DFM Mimir is the narrowest entry: a Danish and English foundation model carrying the hrm_text architecture tag, with 6,477 downloads on the base repo and 383 on its GGUF. The architecture support matters more than this specific model, because HRM-Text is now a path other releases can use.

The MiMo-V2.6 gap between a headline and a download

MiMo-V2.6 was the open-weights story of the week, and "MiMo-V2.6 conversion support" is the line in the release notes most likely to be misread. The MiMo-V2.6 family has three tiers, and they behave completely differently once you try to pull one.

The 9B distill is the one that works. The official ggml-org build went up on 21 September under an MIT license at 9.53 GB for Q8_0, plus a 0.62 GB projector. Combined with a community build from bartowski, the distill has 38,542 downloads across the two repositories. At 10.15 GB it fits a 16 GB card comfortably and a 12 GB card only if you keep context short.

The Flash tier does not work on consumer hardware, and the file sizes are not close. The Flash GGUF is 125.72 GB at Q2_K and 167.37 GB at MXFP4. Q2_K is already an aggressive quantization, and it is still 5.2 times a 24 GB card. Flash has 3,088 GGUF downloads against Muse Glimmer's 711,072, a ratio of roughly 230 to 1, which is a reasonable proxy for how many people got it running.

The Pro tier is the sharpest illustration. MiMo-V2.6-Pro-RL has 4,070 downloads of its original weights. The only Pro GGUF on the Hub was created on 23 September, contains a single 2.94 GB DFlash file rather than the full model, and has zero downloads. "Conversion support" in a release note means the converter understands the architecture. It does not mean somebody has produced, uploaded, and validated a file you can run, and for the Pro tier on 23 September, nobody had.

Qwen4Exp is plumbing for a model that has not shipped

Two of the pull requests in the new-models section add Qwen4Exp hyper-connection ops and sparse flash attention. Search the Hub for Qwen4Exp and you get tiny test fixtures with names like qwen4_exp_tiny_fixture and qwen4-exp-tiny-random-bf16, plus one experimental quantization of the current generation. There is no Qwen4 model to run.

This is normal and it is worth reading correctly. Inference runtimes add architecture support ahead of a release so that day-one weights have somewhere to land, which is why our coverage of the Qwen3.8-Flash-Next release and its Qwen4 preview matters as context here. Treat a Qwen4Exp line in a changelog as a schedule signal about the next model, not as a capability you have today.

The changes that outrank the model list

For most people running llama.cpp in production, three unglamorous items in 0.5.0 will matter more than any of the six models.

The CUDA conv2d implicit GEMM path accelerates convolution, which is what multimodal projectors lean on when they process an image. If your workload is vision, this touches every request rather than one model. The Metal MoE and SSM_CONV fusions do the equivalent on Apple Silicon for mixture-of-experts and state-space models, which describes most current open-weights releases.

The server changes are the ones to read if you self-host. The router got fixes for eviction races and child-process lifecycle handling, and it no longer passes the log file or API key file to router-spawned children. That last item is a credential-scoping fix, not a performance note. The vendored cpp-httplib also moved to 0.57.1.

Multi-address binding is a smaller convenience with real consequences: one server process can now listen on a UNIX socket for local tooling and a TCP address for your network at the same time, instead of running two processes.

How to run the two that fit tonight

If you have a 16 GB or 24 GB card and want the useful half of this release working in under an hour, the path is short.

1. Update to 0.5.0 rather than a nightly build. The nightly attached to this release is b11146, but the tagged version is what the notes above describe.
2. Pull the MiMo-V2.6 9B distill from the ggml-org repository, both the 9.53 GB Q8_0 weights and the 0.62 GB mmproj file. The projector is not optional if you want image input; a text-only load will start fine and then fail on your first image.
3. Pull Muse Glimmer 30B at Q4_K_M, 16.76 GB, plus its 1.4 GB projector. On a 24 GB card, start with a modest context window and raise it until you hit memory pressure rather than guessing.
4. Add --fuse-qkv when you launch Muse Glimmer. This is the flag 0.5.0 added for this model; measure tokens per second with and without it on your own hardware, because the gain depends on your backend.
5. If you process documents, add HunyuanOCR at 0.58 GB. It costs almost nothing to keep resident alongside a larger model.

Compare the result against what the same work costs through a hosted endpoint before deciding what stays local. Our comparison of llama.cpp and LM Studio covers the ergonomics tradeoff if you would rather not manage flags by hand.

Frequently asked questions

Does llama.cpp 0.5.0 let me run MiMo-V2.6 Pro locally?

Not in any practical sense on 23 September 2026. The converter understands the architecture, but the only Pro GGUF published on Hugging Face is a single 2.94 GB DFlash component with zero downloads, not a complete runnable model. The 9B distill at 9.53 GB is the tier of that family you can actually use.

What is the smallest card that runs Muse Glimmer 30B after this release?

The Q4_K_M weights are 16.76 GB and the projector is 1.4 GB, so 18.16 GB before any context. A 24 GB card is the comfortable floor. A 16 GB card will not hold the Q4_K_M file plus a working KV cache.

What does --fuse-qkv actually do?

It fuses the query, key, and value projections into a single operation rather than three, which reduces kernel launches and memory traffic. 0.5.0 adds it for Muse Glimmer specifically. The size of the speedup depends on your backend and batch size, so measure it rather than assuming a number.

Is Qwen4 available in llama.cpp now?

No. 0.5.0 adds Qwen4Exp hyper-connection operations and sparse flash attention, which is architecture groundwork. No Qwen4 weights have been released, and the only Qwen4Exp artifacts on Hugging Face are test fixtures.

Should I upgrade if I only use text models?

Yes, but for the backend work rather than the model list. The ggml v0.25.0 bump, the Metal MoE and SSM fusion optimizations, and the router eviction and credential-scoping fixes apply regardless of which model you load.

Does this release change anything about GGUF quantization itself?

Not directly. 0.5.0 adds architecture and acceleration support, not new quantization formats. If you are weighing GGUF quality tradeoffs, the benchmark data in our writeup of Transformers running GGUF quants is the more useful reference.