Unsloth released new NVFP4 quantized versions of Qwen3.6 on July 10, 2026, that run up to 2.5x faster than other NVFP4 implementations. The headline for anyone running models at home: the 27B variant hits that speedup on a single 24GB GPU, and the 35B-A3B mixture-of-experts model runs on 32GB while keeping tool-calling and agent behavior intact.

Try It: Run Qwen3.6 Faster Locally

If you already run Qwen3.6 through vLLM or SGLang, the change is a model swap. Pull the Qwen3.6-27B-NVFP4 checkpoint for a 24GB card, or the 35B-A3B-NVFP4-Fast build for a 32GB card, and point your existing serving command at it. Unsloth's run guide lists the exact backend flags. The one gotcha: avoid the Marlin backend, which causes a 2.5x slowdown, and use CUTLASS, Flashinfer-TRTLLM, or cute-DSL instead.

Why It Matters for Creators

Local models are only useful in a creative workflow if they respond fast enough to stay in the loop. NVFP4 uses 4-bit weights and 4-bit activations (W4A4), which runs natively on the FP4 tensor cores in NVIDIA's Blackwell GPUs. That is what unlocks the speed: fitting a capable 27B model on a 24GB card at interactive speeds means agent loops, batch rewrites, and code assistance run on your own machine instead of a metered API. The 35B-A3B build is a mixture-of-experts model that activates only about 3B parameters per token, so it delivers larger-model quality at a smaller model's latency, which is exactly the profile you want for a local assistant that has to keep pace with your typing.

Key Details

Speedup: Roughly 2.5x faster inference than standard NVFP4 for the 27B model, with an additional 1.4x to 2.2x from built-in multi-token prediction (MTP) when paired with speculative decoding.

Hardware: 27B on 24GB VRAM, 35B-A3B on 32GB. Both are tuned for Blackwell-class GPUs.

Accuracy: Unsloth uses dynamic per-layer quantization and FP8 KV-cache calibration to hold accuracy while dropping to 4-bit, and reports improved tool calling and agent use over prior quants.

What to Do Next

Grab the checkpoint that matches your GPU from the Unsloth collection and test it against your current setup on a real task before switching over. If you are still choosing hardware for local inference, our guide to running large open models on consumer hardware covers the tradeoffs.