Embedding Models for RAG in 2026: How to Choose, Provider Comparison

You've built a RAG pipeline, connected an LLM, and set up a vector store — but the search returns irrelevant results. The problem is almost always not with the LLM, but with the embedding model. It's the embedding model that determines how accurately the system understands the content of the text and finds the right snippets. Spoiler alert: for an English-language start, use OpenAI text-embedding-3-small or voyage-4-lite at $0.02/1M tokens. For multilingual content, use Qwen3-Embedding-8B ($0.01) or Cohere embed-v4. Below is a guide on when to choose which.

📌 Section 2. Embedding Model Selection Criteria

There's no single "best" embedding model — there's only the best for your specific task, content language, and budget.

Price per 1M Tokens

The range is from $0 (open-source, self-hosted) to $0.20 (Gemini Embedding 2). For a blog with 500 articles, the difference between $0.02 and $0.13 is cents. For a corpus of millions of documents, it's hundreds of dollars monthly.

Vector Dimensionality

384 (MiniLM) → 768 (nomic) → 1024 (mxbai, Voyage 4) → 1536 (OpenAI small, Cohere v4) → 3072 (OpenAI large, Gemini) → 4096 (Qwen3-8B). Higher dimensionality = more accurate search, but more space in the vector store. Most new models support Matryoshka (MRL) — dimensionality can be reduced without re-indexing with another model. For pgvector with a few thousand chunks, the difference is negligible.

Context Window

How many tokens the model processes per request. 512 (mxbai, Nomic V2) — short chunks. 8192 (OpenAI, Gemini, nomic-embed-text) — sufficient for articles. 32K (Voyage 4, Jina v5, Qwen3-8B) — entire documents. 128K (Cohere embed-v4) — the maximum among popular models.

Multilingual Support

Crucial for projects with content in multiple languages. OpenAI works well with English, less so with Cyrillic. Cohere embed-v4 and Gemini Embedding 2 support 100+ languages. Among open-source options, Qwen3-Embedding (a leader in multilingual MTEB) and BGE-M3 (dense + sparse vectors in one model).

Retrieval Quality (MTEB Benchmark)

MTEB (Massive Text Embedding Benchmark) is the standard for comparing embedding models. Note: MTEB v1 and v2 scores are not directly comparable, and the English and multilingual leaderboards yield different rankings. Most importantly, benchmarks don't always reflect quality on your specific data. Test on your own content.

Inference Speed

For indexing (batch, nightly scheduler) — not critical. For real-time search — important. API providers typically respond within 50–200ms. Self-hosted on CPU is slower.

Conclusion: determine your content language, budget, and scale — and these three factors will narrow your choice down to 2–3 models.

📌 Section 3. Paid API Providers

OpenAI: text-embedding-3-small and text-embedding-3-large

text-embedding-3-small — 1536 dimensions, $0.02/1M tokens. text-embedding-3-large — 3072 dimensions, $0.13/1M tokens, supports MRL (can be reduced to 256 with minimal quality loss). Both models are from 2024: OpenAI hasn't released new embedding models since then, so they already lag behind newer competitors on benchmarks. However, they offer the simplest integration and the widest ecosystem.

Suitable for: general RAG, English-only content, quick start. Not suitable for: multilingual content with Cyrillic, code, and technical documentation.

Cohere: embed-v4

100+ languages, 128K token context, $0.12/1M. Defaults to 1536 dimensions (can be scaled from 256–1536 via MRL). Multimodal: text and images in the same space. Binary quantization reduces storage by ~90% with minimal quality loss.

Suitable for: multilingual products, long documents, enterprise use. Not suitable for: English-only content with a limited budget — there are cheaper options.

Voyage AI: Voyage 4 Series

Three models with 32K context and 1024 dimensions by default (256–2048 via MRL): voyage-4-large — $0.12, highest accuracy; voyage-4 — $0.06; voyage-4-lite — $0.02, similar to OpenAI small, but with four times the context. Vectors from the entire series are compatible. There are separate models for code, finance, and law. Recommended by Anthropic.

Suitable for: technical documentation and code (large), budget RAG with long chunks (lite). Not suitable for: if you need multimodality or self-hosting.

Google Gemini Embedding 2

Multimodal model: text, images, video, audio, and PDF in a single vector space. $0.20/1M tokens ($0.10 via batch API), 8192 context, up to 3072 dimensions — Google recommends 768 as the optimum for quality and size. One of the strongest commercial APIs on benchmarks. Vectors are incompatible with gemini-embedding-001.

Suitable for: multimodal search, Google Cloud ecosystem. Not suitable for: if you only need text — it's the most expensive on the list.

Jina AI: jina-embeddings-v5

Two text models: v5-text-small (1024 dimensions) and v5-text-nano (768 dimensions, runs even on CPU). 32K context, 90+ languages, task-specific LoRA adapters for retrieval, matching, clustering, and classification. Via API — $0.05/1M. Weights are open-source but under a CC-BY-NC license — commercial self-hosting requires a separate license.

Suitable for: multilingual search at a moderate price, edge deployment (nano). Not suitable for: free commercial self-hosting.

Conclusion: from my experience, for a quick start, OpenAI small is the easiest to integrate. If you need longer context for the same price, go with voyage-4-lite. For multilingual content, use Cohere. For code and technical docs, use voyage-4-large. For images, video, and audio, use Gemini. For multilingual capabilities at a moderate price, Jina.

📌 Section 4. OpenRouter: Embeddings via a Unified API

Available Embedding Models on OpenRouter

openai/text-embedding-3-small — $0.02/1M, 1536 dimensions. openai/text-embedding-3-large — $0.13/1M, 3072 dimensions. qwen/qwen3-embedding-8b — $0.01/1M, 32K context, multilingual. voyage/voyage-4 — $0.06/1M, 32K context. google/gemini-embedding-2 — $0.20/1M, multimodal. perplexity/pplx-embed-v1-0.6b — $0.004/1M, 32K context — the cheapest option.

When OpenRouter is the Right Choice

• ✔️ Need to quickly compare several embedding models on the same data
• ✔️ Project already uses OpenRouter for LLMs (single API key and billing)
• ✔️ Need a fallback: if one model hosting goes down, the request goes to another

When It's Better to Go Directly to the Provider

• ⚠️ Need specific provider features (Cohere binary quantization, Jina LoRA adapters)
• ⚠️ Enterprise SLA and availability guarantees
• ⚠️ Large volumes — the aggregator's commission becomes noticeable

For local development with Ollama, embedding models run for free without any API. More details — Ollama in 2026: What It Is and Why Developers Are Massively Moving to Local AI.

Conclusion: for me, OpenRouter is a convenient aggregator for comparing models and getting started quickly. For production with specific requirements, I recommend using the provider's direct API.

📌 Section 5. Open-Source Models (Self-Hosted)

Qwen3-Embedding (0.6B, 4B, 8B)

Three sizes for different hardware: 0.6B (1024 dimensions), 4B (2560), 8B (4096). 32K context, 100+ languages, Apache 2.0 license, MRL support. The 8B model leads multilingual MTEB among models that can actually be run locally. Available in Ollama: qwen3-embedding:8b takes up ~4.7 GB, 0.6b — ~640 MB.

BGE-M3 (BAAI)

Hybrid search: dense + sparse vectors in one model. 100+ languages, 8192 context. Ideal for RAG where both semantic and keyword search are needed simultaneously. As described in the article RAG with Ollama, falling back from vector search to keyword search is a critical production pattern, and BGE-M3 handles this at the model level.

Nomic Embed Text V2

Mixture-of-Experts: 475M parameters, with only 305M active — fast for its class. ~100 languages, 768 dimensions (can be up to 256). But only 512 tokens context — only for short chunks. For long texts in Ollama, the first version of nomic-embed-text (8192 tokens) is better.

EmbeddingGemma and MiniLM

Lightweight models for low-end hardware and prototypes. EmbeddingGemma (~300M) is the best multilingual model of its size, 100+ languages. MiniLM — 46 MB, 384 dimensions, English only. Both are inferior to Qwen3 and BGE-M3 on retrieval tasks but run even on CPU.

Pros and Cons of Self-Hosting

• ✔️ Free — zero API costs
• ✔️ Full data control — nothing goes to the cloud
• ✔️ No rate limits or dependency on external services
• ⚠️ GPU required for fast generation (slower on CPU)
• ⚠️ Infrastructure is on you: updates, scaling, monitoring
• ⚠️ Check the license: not all open weights allow commercial use

Conclusion: open-source is the right choice for confidential data, minimal budget, or full control. For most projects, use Qwen3-Embedding (size depends on your hardware) or BGE-M3.

📌 Section 6. Comparison Table of All Models

Conclusion: the table is a starting point. The final choice is only after testing with your real data.

📌 Section 7. Who is it for? — A Choice Matrix

Conclusion: identify your situation in the table — and you'll get a specific recommendation to start with.

💼 Section 8. My Experience: Embeddings for WebCraft's Multilingual Blog

I found most embedding issues not in the documentation — but through real user queries where the search returned irrelevant results.

Stack

Java 21, Spring Boot, Spring AI, PostgreSQL with pgvector on Railway. Embeddings: openai/text-embedding-3-small via OpenRouter. LLM for generation: meta-llama/llama-3.3-70b-instruct (free version). Indexing: nightly scheduler, batches of 10–20 documents.

Why I Chose text-embedding-3-small

$0.02 per million tokens — for ~2000 documents, this is literally cents per month. 1536 dimensions — sufficient "resolution" for blog content. OpenAI-compatible API via OpenRouter — minimal code changes when switching models.

Problems I Encountered

Cyrillic: Search quality for Ukrainian queries is noticeably lower than for English ones. A query like "як зробити сайт" (how to make a website) returned less relevant results than the equivalent "how to build a website". This is critical for a multilingual blog.

Short Queries: Queries with 1–2 words (e.g., "RAG" or "Spring") produced fuzzy results with low scores. The solution: fallback to keyword search for short queries, as described in the article on RAG with Ollama.

Duplicates During Re-indexing: Without deterministic chunk IDs, the database accumulated duplicates. The solution: a UUID based on document_id + chunk number.

What I Would Do Differently

For multilingual content with Cyrillic, I would consider Qwen3-Embedding-8B ($0.01/1M — cheaper than OpenAI small) or Cohere embed-v4. Both are noticeably better with non-Latin languages. Important: changing the model means a full re-index, so it's best to compare models on a sample of real queries first, not the entire database.

Real Numbers

• ✔️ ~500 articles × 4 languages = ~2000 documents
• ✔️ ~10,000 chunks of 512 tokens each
• ✔️ Indexing: ~15–20 minutes via API
• ✔️ Embedding Cost: less than $1 per month
• ✔️ Search: 100–300ms per query

A full breakdown of RAG architecture for production — RAG in 2026: From PoC to Production.

Conclusion: For a multilingual blog, OpenAI small works, but not perfectly with Cyrillic. For similar projects, it's worth testing Qwen3-Embedding-8B or Cohere — the price is the same or lower, and the quality for Cyrillic is noticeably higher.

❓ Frequently Asked Questions (FAQ)

Can I change the embedding model without re-indexing?

No. Different models produce incompatible vectors — even a new version from the same provider (e.g., Gemini Embedding 2 is incompatible with gemini-embedding-001). A rare exception is the Voyage 4 series, where models share a common vector space. Therefore, choosing an embedding model at the start is one of the most crucial architectural decisions in RAG.

Are free models worse than paid ones?

Not always. Qwen3-Embedding-8B and BGE-M3 outperform OpenAI small on multilingual retrieval tasks, and Qwen3-8B is among the leaders on MTEB overall. But benchmarks aren't a guarantee. Test on your own data.

How much does embedding cost for a small project?

For a blog with 500 articles using OpenAI small — less than $1 per month. With Qwen3 via OpenRouter — less than $0.50. Self-hosted — $0, but requires a server with a GPU for fast generation.

Which model should I choose for Ukrainian content?

OpenAI works, but the quality is lower than for English. The best balance of price and quality is Qwen3-Embedding-8B ($0.01/1M on OpenRouter or free locally). Alternatives include Cohere embed-v4 and BGE-M3.

What is Matryoshka Representation Learning (MRL)?

A technique that allows reducing vector dimensionality without retraining the model. For example, OpenAI large generates 3072 dimensions, but you can use only the first 256 — with minimal quality loss. This saves space in the vector store and speeds up search. Today, most new models support MRL: Voyage 4, Cohere embed-v4, Gemini Embedding 2, Qwen3-Embedding.

✅ Conclusions

• 🔹 The embedding model is the foundation of RAG. A poor model will render the entire pipeline useless, regardless of the LLM's quality.
• 🔹 English content: OpenAI text-embedding-3-small or voyage-4-lite ($0.02/1M) — the latter has a 32K context.
• 🔹 Multilingual content with Cyrillic: Qwen3-Embedding-8B ($0.01/1M), Cohere embed-v4, or BGE-M3 — significantly more accurate than OpenAI.
• 🔹 Changing the embedding model = full re-indexing. Choose wisely at the start.
• 🔹 Benchmarks (MTEB) are a useful guide, but the final decision should only be made after testing on your real data and queries.

Main takeaway: I recommend not spending weeks choosing the "perfect" embedding model. Start with OpenAI small (for English content) or Qwen3-Embedding-8B (for multilingual), log the scores on real queries, and only replace the model if you see specific issues with search quality.

If you're just starting with RAG — RAG with Ollama: From Pipeline to Production will give you a complete picture of the architecture. If you need to understand the difference between LLM and RAG — LLM vs RAG in 2026. And a full guide from PoC to production — RAG in 2026: From PoC to Production.