NeoMME: Apache 2.0 Multimodal Encoder for RAG

H Company has released NeoMME, a family of open multimodal and multilingual encoders that read text and raw image patches through a single transformer. Published on September 3, 2026 under the Apache 2.0 license, NeoMME targets a job most builders solve badly today: searching and retrieving across documents that mix text and visuals.

What This Enables

NeoMME is built for visual document retrieval and retrieval-augmented generation. Instead of running a separate OCR pass and a separate text embedder, you feed pages directly to the encoder and get back embeddings you can search. That means a builder can index a library of PDFs, slide decks, or scanned forms and let an agent retrieve the exact page that answers a query, including pages where the answer lives in a chart or a screenshot. A live visual RAG demo shows the retrieval flow end to end.

Why It Matters for Builders

Retrieval quality and cost are the two things that make or break a document-search feature. NeoMME pushes on both. The 260M model sits on the size-versus-quality Pareto frontier and beats every model below 800M parameters on the ViDoRe v3 benchmark, so you do not need a giant model to get strong results. Its late-interaction embeddings compress from roughly 1.5 MB to 6 kB per page, a 255x reduction, while keeping more than 95 percent of retrieval quality. That compression is what makes indexing millions of pages affordable.

Key Details

Sizes: Two models, 260M and 800M parameters, plus retriever and Sentence Transformers variants. See the 800M model card for weights and usage.

Multilingual: A 131k-token vocabulary trained on multilingual text, code, math, and image transcripts, with a 16,384-token context length.

Throughput: The 260M model encodes about 51 pages per second at 2048x2048 resolution on an NVIDIA L40S, roughly twice the speed of ColModernVBERT.

What to Do Next

NeoMME is integrated into Hugging Face Transformers with dedicated processor, model, and retrieval classes, and fine-tuning is supported through Sentence Transformers. If you are building document search or agent memory, start with the 260M retriever, index a sample of your own documents, and measure retrieval accuracy before scaling to the 800M model.