Adaptive RAG: Designing Retrieval Pipelines That Choose the Right Strategy at Runtime

Your RAG system is making the same mistake on every query.

Not a bad mistake. A fixed one.

It retrieves the same number of documents using the same strategy for every question that arrives — regardless of whether that question is a simple factoid lookup, a complex multi-hop reasoning task, or something your model already knows well enough to answer without any retrieval at all.

This fixed-strategy approach is the single largest source of avoidable cost and quality loss in production RAG systems today. And it is entirely architectural. The model is not the problem. The pipeline is.

Adaptive RAG fixes this by answering a question before retrieval begins: what kind of question is this, and what retrieval strategy does it actually need?

This is the complete end-to-end guide to designing retrieval pipelines that make this decision correctly at runtime.

Table of Contents

1. The Fixed-Strategy Problem
2. What Adaptive RAG Actually Is
3. The Query Complexity Taxonomy
4. The Runtime Router: How Strategy Selection Works
5. The Strategy Menu: Six Retrieval Modes
6. Adaptive-k: Choosing How Much to Retrieve
7. Mixture-of-Retrieval-Experts: Adaptive Fusion
8. TARG: Training-Free Adaptive Gating
9. Retriever Portfolios: The Principled Framework
10. Building the Complete Adaptive Pipeline
11. Evaluation and Monitoring
12. Decision Framework

1. The Fixed-Strategy Problem

A production RAG system receives a stream of queries that are radically different in what they require.

"When was the company founded?" requires no retrieval. The model knows this from its training data, and retrieving documents about the company's history adds latency, cost, and context noise without improving the answer.

"What is our current refund policy?" requires single-step retrieval. One well-targeted retrieval pass against the policy document corpus returns the relevant content. Iterative multi-hop retrieval would add unnecessary overhead.

"Which suppliers were impacted by the Q3 logistics disruption and how does that correlate with the delayed shipments reported by enterprise customers?" requires multi-hop retrieval across multiple data sources with intermediate reasoning steps between each retrieval round.

A fixed-strategy RAG pipeline applies one approach to all three. It either over-retrieves on simple queries — adding latency and cost for no quality gain — or under-retrieves on complex queries — returning incomplete context that produces hallucinated or incomplete answers.

The research is unambiguous on this. Adaptive-RAG, published at NAACL 2024 by Jeong et al., demonstrated that routing queries to the cheapest sufficient retrieval strategy — no retrieval, single-step, or multi-step — matches always-expensive multi-hop baselines while substantially reducing cost. Most deployed systems apply one paradigm uniformly to every query. Routing each query to the cheapest sufficient paradigm reduces tokens consumed by orders of magnitude without accuracy loss.

The Retriever Portfolios paper from arXiv:2605.31176, published May 2026, frames this with the precision the field needed: no single retriever is optimal for all queries, and fixing a single retrieval strategy leaves substantial performance on the table across diverse information needs. The practitioner community has moved from asking "which retrieval strategy is best?" to asking "which retrieval strategy is best for this specific query at this moment?"

2. What Adaptive RAG Actually Is

Adaptive RAG is a retrieval architecture where the retrieval strategy — the method, depth, and volume of retrieval — is determined at runtime based on the characteristics of the incoming query rather than fixed at system design time.

It is not one algorithm. It is an architectural pattern with three design decisions:

Decision 1: When to retrieve. Should this query trigger retrieval at all, or can the model answer from parametric knowledge?

Decision 2: What strategy to use. If retrieval is needed, should it be dense vector search, sparse keyword search, hybrid, graph traversal, or iterative multi-hop?

Decision 3: How much to retrieve. How many documents or chunks should the retrieval return? The right answer for a focused factoid question and the right answer for a synthesis task are dramatically different.

Adaptive RAG answers all three questions at runtime. The architecture consists of two stages running before the standard retrieval pipeline: a complexity classifier that characterizes the query, and a strategy router that maps the classification to a retrieval configuration.

The output of this routing stage is not a single retrieved set of documents. It is an instruction to the retrieval subsystem: execute this specific strategy, with these parameters, against these data sources.

3. The Query Complexity Taxonomy

The first step in any adaptive RAG implementation is defining the complexity classes the system must distinguish. The research community has converged on a taxonomy that is practical enough to implement and precise enough to drive meaningful routing decisions.

Class A: No retrieval needed. Queries answerable from the model's parametric knowledge without external grounding. Simple factoids, well-known definitions, historical events that are stable and well-represented in training data. Routing these to the full RAG pipeline wastes compute and introduces context noise from retrieved documents that add nothing to the model's existing knowledge.

Class B: Single-step retrieval. Queries requiring one targeted retrieval pass. Policy lookups, product specifications, procedure documentation, definitions within a specific corpus. The answer lives in one or a small number of documents, and one well-targeted retrieval pass is sufficient to surface it.

Class C: Multi-step retrieval. Queries requiring sequential retrieval where intermediate results determine subsequent retrieval queries. "Which customers were affected by the service outage that was caused by the infrastructure change?" requires finding the infrastructure change, then finding the outage it caused, then finding the affected customers. Each step's output informs the next query.

Class D: Aggregation queries. Queries requiring synthesis across many documents without a clear retrieval chain — "what are the recurring themes in customer feedback this quarter?" The answer is not in any document; it emerges from analysis across the entire corpus.

Class E: Hybrid queries. Queries requiring multiple retrieval modalities — structured data and unstructured text, graph-connected entities and vector-similar content. The strategy must fan out across modalities and synthesize the results.

Getting this taxonomy right for your specific domain is more important than implementing any particular routing algorithm. A five-class taxonomy based on generic research assumptions will underperform a three-class taxonomy calibrated on your actual query distribution.

4. The Runtime Router: How Strategy Selection Works

The router is the core of adaptive RAG. It receives the incoming query and produces a routing decision — which complexity class does this query belong to, and which retrieval strategy should execute.

Three routing approaches have been validated in the research:

Trained classifier routing. A lightweight classifier — T5-Large in the original Adaptive-RAG paper, smaller models in the RAGRouter-Bench study published April 2026 — is trained to predict query complexity class from query text. Training requires labeled examples of queries annotated with their correct complexity class. The classifier is small, fast, and cheap — adding under 100 milliseconds to the pipeline while making routing decisions that save seconds of unnecessary retrieval work.

The RAGRouter-Bench study found that lighter classifiers — logistic regression on sentence embeddings, small transformer classifiers — match the performance of larger T5 classifiers on routing accuracy while being faster and cheaper to deploy. The routing decision itself does not require a large model.

Training-free adaptive gating. TARG — Retrieval as a Decision, arXiv:2511.09803, updated April 2026 — introduces a training-free approach using the model's own confidence signals to decide whether retrieval is needed. If the model's output probability distribution on a query is confident — a small number of tokens have high probability — the model likely knows the answer from parametric knowledge and retrieval is not needed. If the distribution is diffuse, retrieval is triggered.

TARG's key finding: on five QA benchmarks spanning short-answer, multi-hop, and long-form tasks, it consistently matches or improves exact match and F1 over always-retrieving approaches while reducing retrieval frequency significantly. No training data required. No labeled complexity classes. Just the model's own confidence as the routing signal.

Embedding-based similarity routing. For systems with established query histories, routing can be driven by similarity to previously classified queries. A new query is embedded and compared to a library of queries with known complexity classes. If a sufficiently similar query exists in the library with a known classification, the new query inherits that classification. This approach compounds value over time — as the query library grows, routing accuracy improves.

The Router Implementation

from enum import Enum
from dataclasses import dataclass
from sentence_transformers import SentenceTransformer
import numpy as np

class ComplexityClass(Enum):
    NO_RETRIEVAL = "no_retrieval"
    SINGLE_STEP = "single_step"
    MULTI_STEP = "multi_step"
    AGGREGATION = "aggregation"
    HYBRID = "hybrid"

@dataclass
class RoutingDecision:
    complexity_class: ComplexityClass
    retrieval_strategy: str
    k_documents: int
    data_sources: list[str]
    confidence: float

class AdaptiveRouter:
    def __init__(self, classifier_model: str, threshold: float = 0.75):
        self.encoder = SentenceTransformer(classifier_model)
        self.threshold = threshold

    def route(self, query: str) -> RoutingDecision:
        complexity = self._classify_complexity(query)
        return self._map_to_strategy(query, complexity)

    def _classify_complexity(self, query: str) -> ComplexityClass:
        multi_hop_signals = [
            "which", "how does", "why did", "what caused",
            "relationship between", "impact of", "correlation"
        ]
        aggregation_signals = [
            "themes", "patterns", "summarize all", "across all",
            "common", "recurring", "overall"
        ]
        simple_signals = [
            "what is", "define", "when was", "who is"
        ]

        query_lower = query.lower()

        if any(s in query_lower for s in aggregation_signals):
            return ComplexityClass.AGGREGATION
        if any(s in query_lower for s in multi_hop_signals):
            return ComplexityClass.MULTI_STEP
        if any(s in query_lower for s in simple_signals):
            if self._model_likely_knows(query):
                return ComplexityClass.NO_RETRIEVAL
            return ComplexityClass.SINGLE_STEP
        return ComplexityClass.SINGLE_STEP

    def _model_likely_knows(self, query: str) -> bool:
        # In production: call model with low max_tokens,
        # measure output entropy as confidence signal (TARG approach)
        return False

    def _map_to_strategy(
        self, query: str, complexity: ComplexityClass
    ) -> RoutingDecision:
        strategy_map = {
            ComplexityClass.NO_RETRIEVAL: RoutingDecision(
                complexity_class=complexity,
                retrieval_strategy="parametric",
                k_documents=0,
                data_sources=[],
                confidence=0.9
            ),
            ComplexityClass.SINGLE_STEP: RoutingDecision(
                complexity_class=complexity,
                retrieval_strategy="hybrid_search",
                k_documents=5,
                data_sources=["primary_vector_store"],
                confidence=0.85
            ),
            ComplexityClass.MULTI_STEP: RoutingDecision(
                complexity_class=complexity,
                retrieval_strategy="iterative_multihop",
                k_documents=3,
                data_sources=["primary_vector_store", "graph_db"],
                confidence=0.80
            ),
            ComplexityClass.AGGREGATION: RoutingDecision(
                complexity_class=complexity,
                retrieval_strategy="global_search",
                k_documents=20,
                data_sources=["primary_vector_store"],
                confidence=0.75
            ),
        }
        return strategy_map.get(complexity, strategy_map[ComplexityClass.SINGLE_STEP])

5. The Strategy Menu: Six Retrieval Modes

Once the router produces a routing decision, the retrieval subsystem executes the appropriate strategy. Six modes cover the complete space of enterprise retrieval requirements.

Mode 1: Parametric (no retrieval). The query routes directly to the LLM without any retrieval augmentation. Reserved for Class A queries where the model's parametric knowledge is sufficient and reliable. Cost: embedding and retrieval cost eliminated entirely.

Mode 2: Single-step dense retrieval. One vector similarity search against the primary vector store. The standard RAG pipeline. Optimal for Class B queries with clear semantic content. Cost: one embedding call, one ANN search.

Mode 3: Single-step hybrid retrieval. One retrieval pass combining dense vector search with sparse BM25 keyword search, fused through Reciprocal Rank Fusion. Fifteen to thirty percent recall improvement over dense-only retrieval on typical enterprise corpora. Optimal for queries containing both semantic intent and specific terminology.

Mode 4: Iterative multi-hop retrieval. Multiple retrieval passes where each pass is informed by the results of the previous one. The query is decomposed into sub-queries. Each sub-query executes a retrieval pass. The results inform the next sub-query. Continues until the retrieval chain is satisfied or a maximum iteration limit is reached. Optimal for Class C queries requiring sequential reasoning through document chains.

Mode 5: Global aggregation search. Retrieves broadly across the corpus to support synthesis tasks. May use GraphRAG community summaries, document clustering, or high-k vector retrieval with aggressive reranking. Optimal for Class D aggregation queries where the answer emerges from corpus-wide pattern analysis rather than specific document retrieval.

Mode 6: Federated multi-source retrieval. Simultaneously queries multiple data sources of different types — vector stores, knowledge graphs, SQL databases, document repositories — and synthesizes results through a merge step. Optimal for Class E hybrid queries requiring information from multiple modality types.

6. Adaptive-k: Choosing How Much to Retrieve

Even within a single retrieval strategy, the number of documents retrieved — k — should adapt to the query rather than remaining fixed.

DynamicRAG, introduced by Sun et al. 2025, adaptively determines both the ranking and the number of retrieved documents for each query. The core component is a dynamic reranker trained using reinforcement learning, where the quality of LLM-generated responses serves as the reward signal. The reranker learns to select the optimal k for each query by observing which k values produce the best downstream answers.

Cluster-based Adaptive Retrieval — CAR, arXiv:2511.14769, published October 2025 — takes a complementary approach. Instead of training a reranker, CAR analyzes the clustering patterns of query-document similarity distances to determine the natural breakpoint in the similarity distribution. Documents above the breakpoint are included; those below are excluded. The k is determined by the structure of the similarity distribution, not by a fixed parameter.

The intuition behind CAR is correct and important: for a focused, specific query, the similarity distribution has a sharp drop-off after a small number of highly relevant documents. For a broad, ambiguous query, the distribution decays gradually across many documents. The shape of the distribution tells you how many documents the query needs.

The Adaptive-k paper by Taguchi et al. implements a simpler version of this insight: retrieve a large candidate set and then cut at the point where the similarity score drops by more than a defined threshold from the top score. This threshold-based cutoff is implementable without training and provides most of the benefit of learned adaptive-k selection.

def adaptive_k_retrieval(
    query_embedding: list[float],
    vector_store,
    max_candidates: int = 50,
    similarity_drop_threshold: float = 0.15
) -> list[dict]:
    candidates = vector_store.similarity_search_with_score(
        query_embedding, k=max_candidates
    )

    if not candidates:
        return []

    top_score = candidates[0][1]
    cutoff_score = top_score - similarity_drop_threshold

    selected = [
        doc for doc, score in candidates
        if score >= cutoff_score
    ]

    return selected

7. Mixture-of-Retrieval-Experts: Adaptive Fusion

When multiple retrieval strategies run on the same query — dense vector search, sparse BM25, graph traversal — their results must be fused into a single ranked list.

Standard Reciprocal Rank Fusion uses fixed weights. Every retrieval method contributes equally to the fused ranking regardless of which is most appropriate for the current query. This works adequately on average but leaves significant performance on the table for queries where one retrieval method is clearly superior.

MoRE-RAG — Mixture-of-Retrieval-Experts RAG — published in Lecture Notes in Business Information Processing 2026, introduces Bayesian decision theory into the fusion mechanism. It derives optimal weights for each retrieval expert based on how reliable each expert has been on similar queries in the past. A query that looks like it should favor dense retrieval gets heavy weight on the dense retrieval results. A query with specific technical terminology gets heavy weight on the sparse retrieval results.

The key finding: MoRE-Ensemble achieves an 18.82 percent average improvement in NDCG@10 over standard RRF across four BEIR benchmark datasets and industrial maintenance corpora. Critically, only 50 to 200 labeled query-document pairs are needed to learn stable fusion weights — making this practical for industrial deployment under limited annotation budgets.

The Bayesian fusion approach:

import numpy as np
from scipy.special import softmax

class BayesianRetrieverFusion:
    def __init__(self, n_experts: int):
        self.n_experts = n_experts
        self.expert_weights = np.ones(n_experts) / n_experts
        self.query_expert_history = []

    def update_weights(
        self,
        query_embedding: list[float],
        expert_scores: list[float],
        ground_truth_score: float
    ):
        weight_updates = np.array([
            ground_truth_score * score
            for score in expert_scores
        ])
        self.expert_weights = softmax(
            self.expert_weights + 0.01 * weight_updates
        )

    def fuse(
        self,
        expert_rankings: list[list[tuple]],
        query_embedding: list[float]
    ) -> list[tuple]:
        doc_scores = {}
        for expert_idx, ranking in enumerate(expert_rankings):
            weight = self.expert_weights[expert_idx]
            for rank, (doc_id, score) in enumerate(ranking):
                rrf_score = weight / (60 + rank + 1)
                doc_scores[doc_id] = doc_scores.get(doc_id, 0) + rrf_score

        return sorted(doc_scores.items(), key=lambda x: x[1], reverse=True)

8. TARG: Training-Free Adaptive Gating

TARG deserves dedicated coverage because it solves the most expensive part of the adaptive RAG problem — deciding when not to retrieve — without requiring any training data.

The insight is elegant. When a model knows the answer to a question from its parametric knowledge, its output token probability distribution is confident: a small number of tokens have high probability and the distribution is peaked. When the model does not know and would benefit from retrieval, the distribution is diffuse — many tokens have similar probabilities and the model is genuinely uncertain.

TARG uses this confidence signal as the retrieval gate. The model processes the query with a very short maximum token budget — just enough to see whether it generates confidently or uncertainly. If confident, retrieval is skipped. If uncertain, the full retrieval pipeline runs.

On five QA benchmarks spanning NQ-Open, TriviaQA, PopQA, MuSiQue, and ASQA, TARG consistently matches or improves exact match and F1 while reducing retrieval frequency significantly compared to always-retrieving approaches.

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

class TARGGate:
    def __init__(self, model_name: str, confidence_threshold: float = 0.7):
        self.tokenizer = AutoTokenizer.from_pretrained(model_name)
        self.model = AutoModelForCausalLM.from_pretrained(model_name)
        self.threshold = confidence_threshold

    def should_retrieve(self, query: str) -> bool:
        inputs = self.tokenizer(query, return_tensors="pt")

        with torch.no_grad():
            outputs = self.model(**inputs)
            logits = outputs.logits[:, -1, :]
            probs = torch.softmax(logits, dim=-1)

        top_prob = probs.max().item()
        entropy = -(probs * torch.log(probs + 1e-10)).sum().item()

        is_confident = top_prob > self.threshold and entropy < 2.0
        return not is_confident

    def gate(self, query: str) -> str:
        if self.should_retrieve(query):
            return "retrieve"
        return "parametric"

9. Retriever Portfolios: The Principled Framework

The Retriever Portfolios paper from arXiv:2605.31176 provides the most theoretically grounded framework for adaptive RAG published to date. It frames the strategy selection problem as portfolio optimization: given a set of available retrieval strategies with known performance profiles, select the portfolio allocation that maximizes expected retrieval quality for each query.

The portfolio analogy is precise. In financial portfolio theory, you do not put all your capital in one asset. You allocate across assets based on their expected returns and your assessment of which assets are best suited to current market conditions. In retrieval portfolios, you do not commit to one retrieval strategy. You maintain a set of strategies and select the optimal allocation for each query based on query characteristics and expected strategy performance.

The key contribution beyond previous adaptive RAG work is moving from a small fixed menu of strategies — Adaptive-RAG's three options — to a principled approach for selecting among a larger strategy space. Rather than hand-designing a fixed set of strategies, the portfolio framework allows any retrieval configuration to be added to the portfolio, and learns which configurations perform best on which query types from production data.

This framework makes adaptive RAG a system that improves over time rather than remaining static. As production data accumulates about which strategies perform best on which query types, the portfolio weights update and routing decisions improve.

10. Building the Complete Adaptive Pipeline

The complete end-to-end adaptive RAG pipeline integrating all components:

from dataclasses import dataclass
from typing import Optional

@dataclass
class AdaptiveRAGResult:
    response: str
    strategy_used: str
    k_retrieved: int
    routing_confidence: float
    retrieved_documents: list[dict]
    latency_ms: float
    cost_estimate_usd: float

class AdaptiveRAGPipeline:
    def __init__(
        self,
        router: AdaptiveRouter,
        targ_gate: TARGGate,
        vector_store,
        graph_db,
        llm,
        semantic_cache,
        reranker
    ):
        self.router = router
        self.targ_gate = targ_gate
        self.vector_store = vector_store
        self.graph_db = graph_db
        self.llm = llm
        self.cache = semantic_cache
        self.reranker = reranker

    def query(self, query: str, tenant_id: str) -> AdaptiveRAGResult:
        import time
        start = time.perf_counter()

        # Stage 1: Semantic cache check
        cached = self.cache.lookup(query, tenant_id)
        if cached:
            return AdaptiveRAGResult(
                response=cached["response"],
                strategy_used="cache_hit",
                k_retrieved=0,
                routing_confidence=1.0,
                retrieved_documents=[],
                latency_ms=(time.perf_counter() - start) * 1000,
                cost_estimate_usd=0.0001
            )

        # Stage 2: TARG confidence gate
        gate_decision = self.targ_gate.gate(query)
        if gate_decision == "parametric":
            response = self.llm.invoke(query)
            return AdaptiveRAGResult(
                response=response,
                strategy_used="parametric",
                k_retrieved=0,
                routing_confidence=0.9,
                retrieved_documents=[],
                latency_ms=(time.perf_counter() - start) * 1000,
                cost_estimate_usd=0.002
            )

        # Stage 3: Complexity routing
        routing = self.router.route(query)

        # Stage 4: Strategy execution
        documents = self._execute_strategy(query, routing)

        # Stage 5: Adaptive-k reranking
        reranked = self.reranker.rerank(query, documents)
        final_docs = self._apply_adaptive_k_cutoff(reranked)

        # Stage 6: Generation
        context = self._build_context(final_docs)
        response = self.llm.invoke_with_context(query, context)

        # Stage 7: Cache population
        self.cache.store(query, response, tenant_id, final_docs)

        elapsed = (time.perf_counter() - start) * 1000
        cost = self._estimate_cost(routing.retrieval_strategy, len(final_docs))

        return AdaptiveRAGResult(
            response=response,
            strategy_used=routing.retrieval_strategy,
            k_retrieved=len(final_docs),
            routing_confidence=routing.confidence,
            retrieved_documents=final_docs,
            latency_ms=elapsed,
            cost_estimate_usd=cost
        )

    def _execute_strategy(self, query: str, routing: RoutingDecision) -> list:
        if routing.retrieval_strategy == "hybrid_search":
            dense_results = self.vector_store.similarity_search(query, k=20)
            sparse_results = self.vector_store.keyword_search(query, k=20)
            return self._rrf_fusion([dense_results, sparse_results])

        elif routing.retrieval_strategy == "iterative_multihop":
            return self._multihop_retrieve(query, max_hops=3)

        elif routing.retrieval_strategy == "global_search":
            return self.vector_store.similarity_search(query, k=30)

        else:
            return self.vector_store.similarity_search(query, k=10)

    def _multihop_retrieve(self, query: str, max_hops: int) -> list:
        all_docs = []
        current_query = query

        for hop in range(max_hops):
            hop_docs = self.vector_store.similarity_search(current_query, k=5)
            all_docs.extend(hop_docs)

            current_query = self.llm.generate_followup_query(
                original_query=query,
                retrieved_so_far=hop_docs,
                hop_number=hop
            )

            if self._sufficient_context(all_docs, query):
                break

        return all_docs

    def _apply_adaptive_k_cutoff(
        self, reranked_docs: list, drop_threshold: float = 0.15
    ) -> list:
        if not reranked_docs:
            return []
        top_score = reranked_docs[0]["score"]
        cutoff = top_score - drop_threshold
        return [d for d in reranked_docs if d["score"] >= cutoff]

    def _rrf_fusion(self, result_lists: list, k: int = 60) -> list:
        doc_scores = {}
        for result_list in result_lists:
            for rank, doc in enumerate(result_list):
                doc_id = doc["id"]
                doc_scores[doc_id] = doc_scores.get(doc_id, 0) + 1 / (k + rank + 1)
        sorted_ids = sorted(doc_scores, key=doc_scores.get, reverse=True)
        doc_map = {d["id"]: d for lst in result_lists for d in lst}
        return [doc_map[doc_id] for doc_id in sorted_ids if doc_id in doc_map]

    def _sufficient_context(self, docs: list, query: str) -> bool:
        return len(docs) >= 5

    def _build_context(self, docs: list) -> str:
        return "\n\n".join([d.get("content", "") for d in docs])

    def _estimate_cost(self, strategy: str, k: int) -> float:
        base_costs = {
            "parametric": 0.002,
            "hybrid_search": 0.005,
            "iterative_multihop": 0.015,
            "global_search": 0.010
        }
        return base_costs.get(strategy, 0.005)

11. Evaluation and Monitoring

Adaptive RAG systems have a monitoring requirement that static RAG systems do not: you must track not just retrieval quality but routing quality. A system that routes queries incorrectly — sending multi-hop questions through single-step retrieval, or sending simple factoids through expensive iterative search — fails even if each individual strategy performs correctly.

Routing accuracy. The fraction of queries routed to the correct complexity class. Measure by sampling production queries, manually labeling their correct class, and comparing to the router's classification. Routing accuracy below 80 percent signals the classifier needs retraining or the complexity taxonomy needs revision.

Strategy-conditional quality. Answer quality measured separately for each routing class. If multi-hop queries routed to single-step retrieval show quality degradation, the router is under-routing complex queries. If simple queries routed to multi-hop retrieval show no quality improvement over single-step but higher latency, the router is over-routing simple queries.

Cost per routing class. Average token cost and latency per query for each routing class. This is the economic metric that justifies adaptive RAG's engineering investment. The cost difference between no-retrieval and multi-hop retrieval should be 10x or greater — and the routing system should be correctly channeling that expensive path only to queries that require it.

The shadow mode validation pattern. Before deploying adaptive routing to production, run it in shadow mode: route queries using both the adaptive system and the current fixed strategy, compare results, and measure agreement. Measure quality on disagreements — cases where adaptive routing would have chosen differently. This gives you empirical evidence of quality improvement before any production traffic is affected.

12. Decision Framework

Implement adaptive RAG when:

Your query distribution has meaningful complexity variance. Your system handles both simple lookup queries and complex multi-hop synthesis tasks. LLM inference cost is significant at your production query volume. You have the engineering depth to implement routing, monitor it, and maintain it. Your evaluation infrastructure can measure per-strategy quality independently.

Start with TARG before building a classifier. Training-free confidence gating requires no labeled data and no classifier training infrastructure. It immediately captures the highest-value routing decision — no-retrieval for queries the model already knows — with zero training cost. Build the complexity classifier for the remaining retrieval-needed queries after you have validated that parametric routing works correctly for your domain.

Add adaptive-k before adding strategy diversity. The performance gain from retrieving the right number of documents — not too few, not too many — is larger than the gain from using a second retrieval strategy for most production systems. Get adaptive-k working correctly first.

Use MoRE-RAG when you already have multi-retriever infrastructure. Bayesian adaptive fusion is the highest-complexity component in this guide. It is appropriate when you have already deployed multiple retrieval strategies and are observing that fixed-weight RRF is underperforming on specific query types.

Closing Thought

The most important insight in adaptive RAG is one that sounds obvious once you have read it but is missed by most teams: the retrieval strategy is not a property of your system. It is a property of each individual query.

A RAG system that applies one strategy to every query is making a category error. It is pretending that all questions are the same shape when they manifestly are not. Some questions need no retrieval. Some need one fast retrieval pass. Some need sequential multi-hop reasoning through document chains. Some need corpus-wide synthesis.

The research from NAACL 2024 through the Retriever Portfolios paper in May 2026 has established this conclusively: routing each query to the cheapest sufficient strategy matches always-expensive approaches on quality while reducing costs by orders of magnitude.

The pipeline that identifies which shape each question has — and routes it to the strategy built for that shape — is not an advanced optimization for mature systems. It is the correct baseline architecture for any production RAG system serving diverse user queries.

Build the router first. Then tune each strategy in its lane.

Research Sources

• Adaptive-RAG — Jeong, Baek, Cho, Hwang, Park. NAACL-HLT 2024. Pages 7036-7050. Query complexity classifier routing among no-retrieval, single-step, and multi-step strategies.
• TARG: Retrieval as a Decision — Wang et al. arXiv:2511.09803. Updated April 14, 2026. Training-free adaptive gating using model confidence signals.
• Retriever Portfolios: A Principled Approach to Adaptive RAG — arXiv:2605.31176. May 2026. Portfolio optimization framework for retrieval strategy selection.
• Lightweight Query Routing for Adaptive RAG — arXiv:2604.03455. April 2026. RAGRouter-Bench. Lighter classifiers match T5-Large routing accuracy.
• Cluster-based Adaptive Retrieval (CAR) — arXiv:2511.14769. Xu et al. October 2025. Similarity distribution analysis for adaptive-k selection.
• MoRE-RAG: Mixture-of-Retrieval-Experts — Lecture Notes in Business Information Processing 2026. Bayesian adaptive fusion. 18.82 percent NDCG@10 improvement over RRF. 50-200 labels sufficient.
• DynamicRAG — Sun et al. 2025. RL-trained dynamic reranker for adaptive document count selection.
• FAIR-RAG: Faithful Adaptive Iterative Refinement — arXiv:2510.22344. Structured Evidence Assessment module. Dynamic within-iteration adaptivity.
• BalanceRAG — arXiv:2605.20084. Risk-calibrated cascaded retrieval. Statistical guarantees on adaptive routing policies.
• RAGRouter-Bench — Wang et al. 2026. Dataset and benchmark for adaptive RAG routing evaluation.
• Dynamic Context Selection for RAG — arXiv:2512.14313. Multi-retriever fusion and positional bias in adaptive context selection.