필사 모드: Embeddings and Rerankers: What Actually Matters in RAG

• Swapping the Embedding Model Means Rebuilding the Index
• Dimension and Max Length Are the Real Design Values
• Skipping the Prefix Means You Are Using It Wrong
• Multilingual Coverage and Korean Quality Are Different Claims
• A Reranker Is a Different Kind of Object
• What an MTEB Score Does Not Tell You
• Code Examples
• The Order to Decide In
• Try It Yourself
• Series Navigation
• References

Model details were read directly from the Hugging Face pages on 2026-08-12. Model cards and licenses change, so check the original again before you use anything.

Swapping the Embedding Model Means Rebuilding the Index

A generation model can be replaced at any time. An embedding model cannot. The moment you swap it, every vector you have stored becomes meaningless and the entire corpus has to be re-encoded. At a few million documents that is a multi-day job, and retrieval quality sits somewhere between two indexes the whole time.

So the embedding choice has to happen much earlier and much more carefully than the generation choice. The grounds for that decision are not a leaderboard rank but the four numbers and one convention below.

Dimension and Max Length Are the Real Design Values

Dimension is not a quality tier; it is storage cost and search speed. Ten million documents stored as float32 come to roughly 15 GB at 384 dimensions and roughly 41 GB at 1024. Vector database billing and whether the index stays resident in memory are decided right there. A model like Qwen/Qwen3-Embedding-0.6B, whose output dimension can be set anywhere from 32 to 1024, lets you defer that decision to deployment time.

Maximum input length directly determines your chunking strategy. The intfloat/multilingual-e5-large card states that text beyond 512 tokens is truncated. Truncation is not an error — the tail silently disappears — which is why teams so often feed in long paragraphs and then wonder why retrieval fails. BAAI/bge-m3 and Alibaba-NLP/gte-multilingual-base, by contrast, accept up to 8192, so document-level encoding is on the table.

Skipping the Prefix Means You Are Using It Wrong

The intfloat/multilingual-e5-large card states that every input must start with a query or passage prefix, even for non-English text. The same convention appears on nlpai-lab/KoE5. For asymmetric retrieval, the query prefix goes on the question and the passage prefix on the document.

Skip it and the model returns vectors without error while retrieval quality quietly drops. If a newly attached embedding model underperforms, re-reading the usage example on the card word by word is faster than suspecting the scores.

Qwen/Qwen3-Embedding-0.6B uses a different form of the convention. The card states the model is instruction aware and that each query must come with a one-sentence instruction describing the task, reporting roughly a 1 to 5 percent improvement over using none. Alibaba-NLP/gte-multilingual-base states that its usage example needs trust_remote_code=True, which is not a quality issue but an approval to execute repository code, and can collide with an internal security policy.

Multilingual Coverage and Korean Quality Are Different Claims

A sentence about supporting 100 languages means the model processes text in those languages, not that it is good at them. The intfloat/multilingual-e5-large card itself states that low-resource languages may see performance degradation.

There are also models trained further with Korean in mind.

All three inherit the dimension and maximum length of their base model. Choosing nlpai-lab/KoE5 therefore drags the 512-token limit along with it. The dragonkue/BGE-m3-ko card carries over the base model limitation that learning of languages other than Chinese and English is insufficient and additional learning is needed, and states an advantage on longer corpus text over short strings.

Which of these fits your data cannot be settled from a card. There is no substitute for pulling about a hundred queries from your own documents, labeling the correct answers, and measuring.

A Reranker Is a Different Kind of Object

An embedding model turns documents into vectors ahead of time and compares them against a query vector. A reranker takes the query and a document together as a pair and produces a relevance score. It cannot be precomputed, so it never runs over the whole corpus — only over the top candidates the embedding stage narrowed down.

The BAAI/bge-reranker-v2-m3 card states that the output is a relevance score that can be mapped to a float between 0 and 1 through a sigmoid. In other words the raw output is not a probability, so leaving your threshold at 0.5 is wrong. The same card states that enabling use_fp16 speeds up computation with a slight performance degradation.

What an MTEB Score Does Not Tell You

The Qwen/Qwen3-Embedding-8B card carries a self-reported MTEB multilingual score of 70.58 and a No.1 ranking as of June 5, 2025. The operationally important part of that sentence is not the score but the date.

Leaderboard positions change over time, and the value on a card is a snapshot from that moment. More fundamentally, the task mix and language weighting of MTEB differ from the document distribution of your service. Internal wiki search, product description search, and legal document search are different problems, and they do not sort by a single aggregate number.

Benchmark figures on a card are self-reported by the publisher and are not independent evaluations. That is also why this post builds no side-by-side score table across embedding models: when the harness and the prompt convention differ, identically named metrics are measuring different things.

Code Examples

# Example: build embeddings while honoring the prefix convention
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("intfloat/multilingual-e5-large")

queries = ["query: how does carrying over annual leave work"]
passages = ["passage: Unused annual leave may be carried over until March of the following year."]

qv = model.encode(queries, normalize_embeddings=True)
pv = model.encode(passages, normalize_embeddings=True)
print((qv @ pv.T)[0][0])

A reranker is attached only after the candidate set is narrowed.

# Example: apply the reranker to the top candidates only
from sentence_transformers import CrossEncoder

reranker = CrossEncoder("BAAI/bge-reranker-v2-m3", max_length=512)

query = "annual leave carryover rule"
candidates = ["Unused annual leave carries over until March.", "Travel expenses are settled at month end."]
scores = reranker.predict([(query, c) for c in candidates])
print(sorted(zip(scores, candidates), reverse=True))

The Order to Decide In

1. Fix the dimension range your document count and budget can carry.
2. Fix the chunk length, then keep only models whose max input does not truncate it.
3. Confirm the prefix or instruction convention on the card and implement it exactly.
4. Pull a hundred queries from your own documents, label them, and compare candidates directly.
5. Default to a two-stage design: embeddings pick the top 50, the reranker picks the top 5.

Try It Yourself

• ML Training Data Explorer — see how the character of training data carries through to retrieval quality.
• AI Benchmark Collection — check what a benchmark measures and what it does not.
• Browser AI Lab — run a small embedding model yourself to build intuition for similarity.

Series Navigation

• Previous: Choosing Open Text Generation Models by Size Class
• Next: Open Models That Support Korean, and the Cost of Tokenization

References

• Every value in the tables was read directly from that model page on Hugging Face on 2026-08-12. Anything absent from the page is written as not stated.
• The MTEB figures for Qwen/Qwen3-Embedding-8B are self-reported on the card and are not an independent evaluation.
• Read the full license text yourself and put commercial use through legal review.