Advanced Retrieval Patterns: Multi-Query, Multi-Vector, Parent-Document, and Diversified Retrieval

Your recall metric looks healthy. Your answers are still wrong. That gap is the whole reason advanced retrieval patterns exist.

The corpus contains the answer. The index contains the chunk. The retriever returns ten results with respectable similarity scores. And the model still produces a confident, incomplete answer — because the ten results it received were the wrong ten. Not irrelevant, exactly. Just narrow, or coarse, or five paraphrases of the same sentence crowding out the one chunk that would have closed the gap.

The default mental model here is seductive: retrieval is similarity search, so better embeddings fix recall. That model is not wrong so much as incomplete. It treats retrieval as a matching problem when it is actually a context-selection problem. And context selection fails along four independent axes — query ambiguity, document structure, granularity, and redundancy — each with a different fix. Re-embedding everything when your real problem is redundancy burns budget and hides the bottleneck.

The invariant to carry through this article: every advanced retrieval pattern buys coverage, context fidelity, or diversity by paying a specific cost in tokens, latency, index size, or extra model calls. The engineering decision is not which pattern is best. It is which cost you can afford for this corpus and this query distribution.

This assumes you already understand chunk boundaries, overlap, and structural metadata. The question here is what happens after chunking, at the retrieval-strategy layer.

Why Single-Query Retrieval Hits a Recall Ceiling

One query string is one point in embedding space. If the user's phrasing sits far from the chunk that answers the question — different vocabulary, different abstraction level, different framing — that chunk may never enter the candidate set, no matter how well it was indexed. The embedding did its job. The query was simply the wrong probe.

Three distinct failures get lumped under "bad retrieval," and they have different fixes:

Query-side mismatch. The user asks about "reducing inference cost," the corpus says "lowering per-token spend," and the two phrases land in different neighborhoods. The evidence exists; the query never reaches it.

Granularity mismatch. The chunk is too small to carry the answer (a sentence that references a table three paragraphs up) or too large to match the query (a 2,000-token section whose embedding is a blur of five topics). The match fails because the unit of comparison is wrong.

Redundancy collapse. The top-k fills with near-duplicates — five chunks that all restate the same fact in slightly different words. They score well because they are similar to each other and to the query. They displace the one chunk that would have answered the actual question.

Each failure has a different remedy. Multi-query addresses the first. Multi-vector and parent-document address the second. Diversification addresses the third. Applying the wrong remedy — re-embedding the corpus when the real issue is that your top-10 is nine copies of the same paragraph — wastes weeks and leaves the failure intact.

The Decision Axes: Ambiguity, Structure, Granularity, Redundancy

Before comparing techniques, define the rubric. Four axes determine which pattern earns its cost:

Query ambiguity. Does one query string reliably express intent? If users ask short, underspecified questions — "how does billing work" across a corpus with six billing subsystems — a single phrasing cannot cover the intent space. If queries are precise and well-formed, multi-query buys latency for nothing.

Document structure. Is the corpus flat prose, or does it have headings, sections, tables, and parent-child hierarchy worth preserving? Structure determines whether parent-document retrieval has a meaningful parent to return.

Granularity. What unit should be matched for precision versus what unit should be handed to the model for context? These are separable decisions, and conflating them is the most common design error in RAG pipelines.

Redundancy cost. How much of the top-k budget is wasted on near-duplicate evidence, and what does that waste displace? If your corpus has heavy paraphrase — changelogs, FAQ variants, multi-version docs — redundancy is eating your context window silently.

Frame each pattern as a purchase. Multi-query buys coverage with extra model calls and vector searches. Multi-vector buys match precision with index size and a mapping layer. Parent-document buys context fidelity with token budget. Diversification buys evidence breadth with ranking-score loss. You are always paying. The question is whether the thing you are buying is the thing you are missing.

The Layers: Where Each Pattern Actually Operates

The four patterns are not four competing choices in one category. They operate at different layers of the pipeline, and confusing the layers is what makes the taxonomy feel like a menu when it is actually a stack.

Multi-vector indexing and parent-document retrieval are not alternatives. Multi-vector decides how you find the candidate. Parent-document decides what you return once you have found it. You can — and often should — use both. A generated-question index that resolves to a parent section is multi-vector at the match layer and parent-document at the expansion layer, working together.

The four-axis mapping is a diagnostic, not a partition. Query ambiguity points you at the transformation layer. Granularity points you at both the representation layer and the expansion layer, because matching precision and context fidelity are separate problems. Redundancy points you at candidate selection. Structure tells you whether the expansion layer has anything useful to expand into.

Multi-Query Retrieval: Buying Coverage With Extra Calls

The mechanism is straightforward: an LLM rewrites or expands the original query into several variants, each variant retrieves independently, and the results merge into a unique union — commonly via reciprocal rank fusion (RRF), which scores documents by summing 1/(k + rank) across result lists so that documents appearing high in multiple lists rise to the top.

What it fixes: query-side mismatch and underspecified intent. A user asking "how do I handle auth failures" generates variants like "authentication error handling," "token expiration recovery," and "401 response retry logic" — each probing a different region of embedding space. The union covers more ground than any single phrasing.

What it does not fix: granularity mismatch or redundancy. If your chunks are the wrong size, five query variants retrieve five wrong-sized chunks. If your corpus is paraphrase-heavy, five variants retrieve fifteen near-duplicates.

The known limitation is worth stating plainly: generated rewrites can be nearly identical and lack diversity, so the recall gain is smaller than the call count suggests. Ask a model to generate four query variants and you may get four rewordings of the same question. Diversity must be engineered — through explicit instructions to vary abstraction level, through temperature, or through diversity-maximizing selection over candidate rewrites — not assumed.

Cost profile: one extra LLM call per query plus N vector searches before any shared or batched stage. The realized multiplier depends on your implementation. Query generation, the N searches, deduplication, and any reranking do not all scale identically — caching and batching can absorb part of the cost. But the default expectation should be that variant count drives search count, and search count drives latency.

When not to use it: high-precision, low-ambiguity queries where a single well-formed query already lands the right chunk. You pay latency and compute for coverage you did not need.

Multi-Vector and Multi-Representation Indexing: Decoupling Match From Content

Here is the core idea that unifies several techniques that look unrelated: the thing you match against does not have to be the thing you hand to the model.

Multi-vector indexing stores multiple representations of the same source — summaries, propositions, generated questions, sub-chunks — each with its own embedding, all pointing back to the full document. A short summary or a generated question often sits closer to a user query in embedding space than the raw passage does, because it is written in the register of a question rather than the register of an answer.

This is the shared substrate under parent-document retrieval, hypothetical-question indexing, and summary indexing. Naming the substrate matters because it prevents treating them as unrelated tricks. They are all instances of decoupling the retrieval representation from the generation representation.

But "multiple representations" is a general pattern, not a single technique. The representation you choose changes the operational profile:

• Generated questions match the query register directly. They are strong for FAQ-style corpora but can overfit to anticipated phrasings and miss unanticipated ones.
• Summaries compress a section into a queryable gist. They are cheap to generate but lose specificity — a summary of a long section may match many queries weakly rather than one query strongly.
• Propositions decompose content into atomic factual statements. They match precisely but multiply index size and require careful deduplication.
• Child chunks are the smallest representation and the one parent-document retrieval builds on. They match precisely but carry no context on their own.

Each choice has a different freshness risk, storage cost, and semantic failure mode. The mechanism in practice: at index time, an LLM generates the chosen representation for each chunk. Those representations get embedded and stored alongside a pointer to the original. At query time, you match against the generated representations, then resolve the pointer to retrieve the full source.

Cost profile: extra LLM calls at index time, a larger index (multiple vectors per source), and a mapping layer that must stay consistent when documents change.

Failure mode — stale representations: if the source document is updated but the summary is not regenerated, retrieval matches a description of a document that no longer exists. The model receives a pointer to content that contradicts the representation that retrieved it. This is a silent failure: the retrieval looks correct, the answer is wrong, and nothing in the logs flags it. Any multi-representation index needs a regeneration trigger tied to source updates, not a one-time build.

Parent-Document Retrieval: Small to Match, Large to Answer

Parent-document retrieval makes the granularity split explicit: index small child chunks for similarity search, but return the larger parent document or section to the generator.

The invariant: retrieval precision and generation context are different objectives and should be tuned separately rather than compromised into one chunk size. Small chunks match precisely because their embeddings are focused. Large chunks answer well because they carry surrounding context. Pick one size and you compromise both.

Three common variants:

• Child-to-parent: index child chunks, return the parent section. The child matches the query; the parent gives the model room to reason.
• Sentence-window: retrieve a single sentence, return its surrounding window of N sentences. Maximum match precision, bounded context expansion.
• Question-to-parent: index generated questions the document answers, return the parent. This combines multi-representation indexing with parent retrieval — the question matches the query register, the parent provides the answer context.

Cost profile: modest index overhead and a parent-lookup step. The real cost is token budget. Parents are larger, so fewer of them fit in the context window. If your child-to-parent ratio is 1:10, retrieving five children may pull in 50x the tokens you budgeted for.

Failure mode — oversized parents: if the parent is a whole chapter, you have traded a precision problem for a noise problem. The model now has the answer somewhere in 4,000 tokens of surrounding material, and attention dilution does the rest. Parent size should be the smallest unit that reliably contains the answer's dependencies — typically a section, not a document.

When not to use it: corpora where chunks are already self-contained and the answer never depends on surrounding structure. API reference docs with one endpoint per chunk do not need parent retrieval.

Diversified Retrieval: Spending the Top-k Budget on Distinct Evidence

Similarity ranking is greedy for near-duplicates. Five paraphrases of the same fact occupy five slots and displace the one chunk that would have closed the evidence gap. The ranking is not wrong — those five chunks genuinely are similar to the query. The ranking is optimizing for the wrong objective: maximum similarity rather than maximum evidence coverage.

Diversified retrieval applies a diversity constraint during or after ranking. Three common mechanisms:

• Maximal marginal relevance (MMR): iteratively select the document that maximizes a weighted combination of query similarity and dissimilarity from already-selected documents. The tradeoff parameter controls how much diversity you buy.
• Per-document or per-section caps: limit how many chunks from any single source can enter the final set. Crude but effective when redundancy correlates with source.
• Cluster-then-select: cluster candidates by embedding similarity, then pick one representative per cluster. Guarantees coverage across distinct topics.

The tradeoff is explicit and worth stating without euphemism: you deliberately accept a lower average similarity score in exchange for broader evidence coverage. That is a loss in ranking metrics and a gain in answer completeness.

This is the pattern most likely to be skipped because it makes offline relevance metrics look worse while improving end-to-end answers. If your evaluation only measures average similarity of retrieved chunks, diversification will appear to hurt. Measure the right thing.

Failure mode — over-diversification: pushing out the single most relevant chunk in the name of variety. Diversity should be a constraint, not the objective. If the answer lives in one chunk, a diverse set that excludes it is worse than a redundant set that includes it.

A Worked Trace: One Query Through Four Stages

Abstract guidance about coverage and redundancy is hard to act on. Here is a compact trace. The query is "how do I handle auth failures." The corpus has eight candidate chunks, three of which are answer-bearing (A1, A2, A3), three redundant (R1, R2, R3 — paraphrases of A1), and two irrelevant (I1, I2).

Baseline single-query top-5: R1, R2, A1, R3, I1. Recall of answer-bearing chunks: 1 of 3. The retriever found A1 and then filled four slots with paraphrases and noise. This is redundancy collapse.

Multi-query union (three variants): R1, R2, A1, R3, I1, A2, I2. Recall: 2 of 3. The second variant reached A2, which the original phrasing missed. Coverage improved, but the set is now larger and still redundancy-heavy.

Diversification (per-source cap of 2, then MMR): A1, R1, A2, I2, A3. Recall: 3 of 3. The cap forced R2 and R3 out, making room for A3. Average similarity dropped — A3 scores lower than the paraphrases it replaced — but evidence coverage is complete.

Parent expansion: each of the five survivors resolves to its parent section. Token count rises from roughly 1,200 to roughly 4,500. The model now has the surrounding context for A1, A2, and A3, including the table that A2 references.

The trace shows three things at once. Recall, context tokens, and answer completeness move independently. Diversification looks worse on a similarity metric while being the stage that closed the gap. And parent expansion is where the token bill arrives — after the candidate set is already correct.

Combining Patterns Without Paying Twice

Patterns compose across layers. Multi-query operates at query transformation. Multi-vector operates at match representation. Diversification operates at candidate selection. Parent-document operates at context expansion. Pick one pattern per layer where you have an actual, observed problem.

A common defensible stack: multi-query for coverage, multi-vector for match precision, diversification as a final constraint, parent-document for context fidelity, with reranking between retrieval and generation. Each stage has a distinct job. None duplicates another.

But each added stage compounds latency and cost. The marginal stage must be justified by a measured failure it removes, not by the fact that it exists. I have seen pipelines with five retrieval stages where the ablation showed three of them contributed nothing measurable — they were added because they were available, not because they solved an observed problem.

Ordering matters, and the answer changes depending on where you place each stage. Diversify before you rerank, and you protect coverage — the reranker sees a diverse candidate set and picks the best from each region. Diversify after you rerank, and you protect precision — the reranker picks the top candidates, then diversification thins near-duplicates from that set. Decide based on whether your failure mode is missing evidence or buried evidence.

Overkill signal: if a single-query hybrid search with reranking already meets your evaluation bar, the advanced stack is latency you are paying for a problem you do not have. Advanced patterns are not a maturity ladder. They are specific tools for specific failures.

Evaluating Which Pattern Actually Helped

The comparison only becomes a decision when you can measure it. The measurement boundary matters more than the metric list, because parent-document and multi-vector systems produce several distinct events that a single recall number conflates.

Define four checkpoints and measure each separately:

1. Representation hit. Did the query match a representation that points at answer-bearing content? This is the match-layer event.
2. Source resolution. Did the pointer resolve to the correct parent or source document? A representation can match while the mapping layer is stale or wrong.
3. Final-context inclusion. Did the answer-bearing content survive into the context the model actually received, after diversification, caps, and budget truncation?
4. Answer support. Did the model's answer cite or use that content? This is the generation-layer event.

A pipeline can pass checkpoint 1 and fail checkpoint 3 — the representation matched, but diversification or a token budget pushed the parent out. It can pass checkpoint 3 and fail checkpoint 4 — the evidence was in context and the model ignored it. Those are different bugs with different fixes.

Two categories of metrics map onto these checkpoints. Retrieval metrics — recall@k, evidence coverage, mean reciprocal rank — measure checkpoints 1 through 3. Answer metrics — answer correctness, citation grounding, completeness — measure checkpoint 4.

Build a small labeled set of queries with known answer-bearing chunks. Then ablate one pattern at a time, holding the downstream generator and the context token budget constant so the comparison isolates the retrieval change. The ablation is the experiment that turns a feature catalog into a decision.

Watch for the diversification paradox specifically: a pattern can lower average similarity while raising answer completeness. A single relevance score will mislead you into removing the pattern that was doing the most work.

Track cost and latency alongside quality. A pattern that adds 40% latency for a two-point recall gain may not survive contact with production traffic. And log the retrieved set per query so failures are diagnosable: was the evidence missing, present but buried, or present and ignored? Those three diagnoses point to three different fixes.

The Decision Rule

Diagnose which layer is actually failing before adding a pattern. Query ambiguity points at the transformation layer. Granularity points at both the representation and expansion layers. Redundancy points at candidate selection. Structure tells you whether expansion has anything useful to expand into.

Add one pattern per failing layer. Measure its marginal contribution with an ablation that holds the generator and context budget constant. If the ablation does not show a gain on the checkpoint that matters for your failure mode, remove it.

The next action is concrete: take your current pipeline, build a small labeled query set with known answer-bearing chunks, and run one ablation on the pattern that maps to the failure you can actually observe in your logs. Not the pattern that sounds most sophisticated. The one that addresses the evidence gap you can point to.