1536 vs 3072 embeddings: comparison for document search and RAG

When a team builds a RAG pipeline and retrieval returns irrelevant results, the typical first reaction is: "we need a larger embedding model." The logic is understandable: more dimensions → more information → better search. But the data says otherwise.

This article is a technical breakdown for developers designing RAG under specific constraints of RAM, cost, and quality. We look at the numbers: MTEB benchmarks, storage costs, latency, and compare alternatives — particularly reranking, which in most scenarios provides a greater increase in retrieval quality at a lower cost than switching to a 3072-dimensional model.

If you are still at the stage of choosing a document parsing approach before vectorization, first read the previous articles in the series: "How OCR Affects RAG System Quality" and "Vision RAG vs OCR: Comparing Approaches". Poor input data nullifies any advantage from a better embedding model.

What is an embedding vector and what does its dimensionality mean

An embedding model converts text into a numerical vector — a list of fixed-length numbers. This vector is the "coordinate" of the text in an n-dimensional semantic space. Texts with similar meanings have close vectors (high cosine similarity); texts with different meanings are far apart.

Vector dimensionality is the number of numbers in this list. text-embedding-3-small returns 1536 numbers per input text. text-embedding-3-large returns 3072.

What dimensionality measures

Important: dimensionality is a property of the model's architecture and training, not simply "more is better." Research on production RAG systems shows that reducing from 1536 to 384 dimensions in some scenarios does not result in a measurable drop in retrieval accuracy — while halving latency and reducing storage by 75%.

Matryoshka Representation Learning (MRL): use large, store less

text-embedding-3-large and text-embedding-3-small support MRL — an approach where the first dimensions of the vector carry the most general semantic signal, and each subsequent one adds detail. This allows "truncating" the vector after generation without full retraining of the model.

In practice: you generate a 3072-dimensional vector via API, but store only the first 256, 512, 1024, or 1536 dimensions using the dimensions parameter. Quality does not drop proportionally — it's non-linear:

Reducing from 3072 to 1536 dimensions provides a −50% RAM reduction with only a 1.6% quality loss. Down to 1024 — a −66% RAM reduction with a −2.8% quality loss. This is the most practical compromise for most scenarios.

Important: MRL only works with models explicitly trained with this technique (OpenAI's text-embedding-3-*, some Cohere and Nomic models). Simply "truncating" a vector from an arbitrary model will result in degradation without guarantees.

How dimensionality affects semantic search quality

The theoretical advantage of higher dimensionality is not always realized uniformly. The impact depends on three factors: the nature of the documents, the type of queries, and the presence of other pipeline components.

Where dimensionality truly matters

Where dimensionality hardly matters

text-embedding-3-small (1536) vs text-embedding-3-large (3072): real differences

MTEB benchmark: the numbers

MTEB (Massive Text Embedding Benchmark) is a standard benchmark for comparing embedding models. For RAG, the most relevant metric is MTEB Retrieval (nDCG@10).

Source: ICLERB Benchmark, MTEB Leaderboard November 2024.

An 8.5% nDCG@10 increase is a real difference, but it's important to understand the context: this is a benchmark on a general set of tasks. On your specific corpus, the difference may be larger or smaller. And for 6.5× higher embedding cost plus 2× higher storage cost.

Market context: OpenAI is no longer the leader

text-embedding-3-large has not been updated since January 2024. According to the MTEB Leaderboard April 2026, the landscape has significantly changed:

Qwen3-Embedding-0.6B with 1024 dimensions (70.70 MTEB) surpasses text-embedding-3-large with 3072 dimensions (66.43 MTEB) — with lower dimensionality and full self-hosted availability. This echoes the main message of the article: dimensionality ≠ quality.

Practical difference on real tasks

Impact on RAM, Disk Space, and Search Speed

The cost of storing vectors is linear with dimensionality. Each float32 takes 4 bytes. 1536 dimensions × 4 = 6,144 bytes (~6 KB) per vector. 3072 dimensions is exactly double.

Specific Numbers for Typical Collections

Source: Qdrant documentation; arXiv 2505.00105 — Optimization of embeddings storage for RAG.

Managed Vector DB Costs at Scale

According to Vector DB Costs 2026 analysis, one embedding call to text-embedding-3-large generates a 3072-dimensional vector — 10M documents yield 120 GB of raw vector data before indexing. Management costs for Pinecone for 10M vectors with 3072 dimensions are ~$70–120+/month versus $35–65 for 1536 dimensions.

For self-hosted Qdrant: 1M vectors of 1536 dimensions require a minimum 4GB RAM tier (~$36/month on Qdrant Cloud). 3072 dimensions for the same volume require an 8GB RAM tier, ~$72/month.

Search Speed (ANN Latency)

Practical Advice: pgvector and 3072 Dimensions

If you are using pgvector, pay attention to an important limitation: GCP PostgreSQL does not allow creating HNSW indexes for vectors with >2000 dimensions. text-embedding-3-large with 3072 dimensions on pgvector in GCP means brute-force search without an index — a catastrophic degradation of latency on collections of 100,000+ chunks. Either reduce to 1536 via the API's dimensions parameter, or switch to Qdrant/Weaviate.

Comparison on Typical RAG Scenarios: Legal Documents, Technical Documentation, FAQs

Scenario 1: Legal Documents

Legal texts are a complex case for dense retrieval. They contain both precise numerical values (sums, dates, articles) and subtle semantic differences (responsibility vs. obligation).

Key takeaway for legal documents: the difference between 1536 and 3072 in the dense component is minimal compared to the effect of adding BM25 in a hybrid pipeline. Research on financial documents from T2-RAGBench (2026) shows that hybrid + reranker yields Recall@5 = 0.816 versus 0.587 for pure dense retrieval with text-embedding-3-large — a +39% improvement without changing the embedding model.

Scenario 2: Technical Documentation

Technical documentation (API references, specifications, RFCs) contains highly specialized terminology. Here, a 3072-dimensional model provides a more noticeable effect — narrow terms are encoded more precisely in a larger space.

However, there's an important nuance here too: if the documentation contains exact names (functions, parameters, versions), BM25 or hybrid remains the better option for precise queries. The dense component helps with "how to do X" queries.

Scenario 3: FAQs and Helpdesk

FAQs are the simplest scenario for embedding models. Queries and answers are semantically straightforward. For FAQ and helpdesk systems, even a 384-dimensional model provides acceptable quality — the difference between 1536 and 3072 on this data is minimal.

For FAQs, the optimal strategy is to start with text-embedding-3-small or even BGE-small (384 dimensions), measure Recall@5 on a test set of questions, and only if there's a measurable deficit, move to a larger model.

Overall Table by Scenario

Why Model Quality Matters More Than Dimensionality

Comparing text-embedding-3-small (1536) and text-embedding-3-large (3072) is a comparison of two products from the same vendor. But the real picture is broader: there are models with lower dimensionality and higher quality.

The already mentioned Qwen3-Embedding-0.6B (1024 dimensions, MTEB 70.70) outperforms text-embedding-3-large (3072 dimensions, MTEB 66.43) on the overall MTEB benchmark with lower dimensionality. Apache 2.0, fully self-hostable.

Factors Determining Embedding Quality Beyond Size

The "FAISS Hybrid Paradox": When a Larger Model is Worse

The "Rethinking Hybrid Retrieval" study (2025) revealed a counterintuitive result: MiniLM-v6 (a compact model) consistently outperforms BGE-Large when integrated with LLM-based reranking in tri-modal hybrid retrieval. The difference: up to 23.1% better nDCG@10 and 36.5% better nDCG@1 in the financial domain — with 93% fewer parameters and 63% smaller embeddings.

The reason is the "FAISS Hybrid Paradox": the embedding space of large models may not align with the relevance criteria of an LLM reranker. A larger model creates a more "stretched" space that the LLM reranker re-ranks less effectively than a compact one.

Conclusion: if your pipeline includes a reranker, it's not guaranteed that a larger embedding model will improve results. Test on your own data.

Reranking as an Alternative to Increasing Dimensionality

A reranker (or cross-encoder) is the second stage of retrieval. In the first stage, an embedding model finds top-k candidates based on cosine similarity (fast, but not precise). In the second stage, the reranker re-ranks these candidates, evaluating each (query, document) pair together (slow, but much more precise).

Why a Reranker is More Precise Than an Embedding

A bi-encoder (embedding model) encodes the query and document independently. This allows for fast ANN search, but means the model does not "see" the query and document simultaneously during encoding.

A cross-encoder (reranker) encodes the (query + document) pair together. Self-attention "sees" both texts simultaneously and can evaluate subtle semantic interactions. This is computationally much more expensive — but also much more precise.

Real Numbers: What a Reranker Provides

Hybrid + reranker yields Recall@5 = 0.816 versus 0.587 for pure dense — a +39% improvement without changing the embedding model. For comparison, switching from text-embedding-3-small to large provides ~8.5% improvement on MTEB Retrieval.

Popular Reranker Solutions

When a Reranker Isn't Enough

A reranker only re-ranks what made it into the top-k from the first stage. If a relevant chunk didn't even make it into the top-50 candidates, the reranker won't help. In this case, a better embedding model or hybrid retrieval with a larger k is needed.

Also important: a reranker adds latency (~100–500 ms for 50 candidates). For real-time systems with end-to-end requirements of <500 ms, this can be a limitation.

Matrix Selection Logic: Collection Size × Accuracy × Budget

Decision-Making Algorithm

FAQ: Frequently Asked Questions About Embedding Dimensionality

Is it worth switching from 1536 to 3072?

In my experience, in most cases, no, at least not as a first step. Switching from small to large provides ~8.5% MTEB Retrieval gain but costs 6.5× more via API and twice as much for storage. Before changing the model, I always check first: is hybrid retrieval (BM25 + dense) implemented, is there a reranker. These two components together provide +39% Recall@5 — without changing the dimensionality at all. If the quality deficit remains after this — then yes, switching to 3072 or a better model is justified.

How much does RAM consumption increase?

Exactly double — the dependency is linear. 1M chunks at 1536 dimensions occupy ~6.1 GB RAM, at 3072 — ~12.2 GB. For 10M chunks, this is already the difference between 61 GB and 122 GB — effectively a different server tier and a different infrastructure budget. For small collections (up to 100K chunks), the difference is not critical: 600 MB vs 1.2 GB fits into any modern instance. But at scale, it becomes the first limitation.

Does dimensionality affect pgvector speed?

It does, and there's an important practical pitfall. Firstly, higher dimensions mean more operations when calculating cosine similarity — query latency increases. Secondly, and this is critical: GCP PostgreSQL does not support creating HNSW indexes for vectors with more than 2000 dimensions. If you use text-embedding-3-large (3072) with pgvector on GCP — you get brute-force search without an index, which on collections of 100K+ chunks leads to catastrophic latency degradation. Solution: either reduce to 1536 via the dimensions parameter in the API, or switch to Qdrant or Weaviate.

Are 3072 dimensions needed for small RAG?

No. For collections up to 50K chunks, the difference between 1536 and 3072 is practically unnoticeable — any model will find the correct chunk in a small index. I recommend starting with text-embedding-3-small or even BGE-small (384 dimensions), measure Recall@5 on a test set of questions, and only then make a decision. Most teams I've worked with found that the problem wasn't dimensionality, but the quality of the input text or the lack of a reranker.

What is more important: model or dimensionality?

The model is always more important. The best example: Qwen3-Embedding-0.6B with 1024 dimensions scores 70.70 MTEB, while text-embedding-3-large with 3072 dimensions scores 66.43. Smaller dimensionality, better result. In my experience, the priority order for RAG optimization looks like this: first, input text quality, then chunking strategy, then hybrid retrieval and reranker — and only after that choosing or changing the embedding model. Dimensionality is the last variable worth tweaking.

Conclusions

The choice between 1536 and 3072 dimensions is not the main decision when designing a RAG pipeline. It ranks much lower in priority than input data quality, chunking strategy, and the presence of hybrid retrieval with a reranker.

Key Takeaways