Evaluating RAG Systems

“We put our documents in a vector store, wired up a GPT-5.5 Instant prompt with context, and shipped it. It was working great in demo.” If that sentence makes you wince, you have probably already run a RAG system in production. The demo is always persuasive: you ask a question, the relevant chunk appears in the context, and the model gives a crisp answer. What you do not see in the demo is the other 80 percent of queries where the retrieval pulls the wrong chunk, or the model ignores the chunk and answers from its own training data, or the chunk is right but only covers half of what the user actually asked.

RAG — retrieval-augmented generation — is the most common pattern in applied AI right now, and for good reason. It lets us ground LLM outputs in our own data without retraining. But RAG is a compound system, and compound systems fail in compound ways. Evaluating one piece in isolation tells you nothing about the whole. Evaluating the whole without instrumenting each piece tells you nothing about what broke.

This article walks through exactly how to evaluate a RAG system: which metrics matter for retrieval, which matter for generation, how to build an evaluation dataset, what good scores look like, and how to use the results to debug failures.

Why RAG Evaluation Is Two Evaluations

A RAG system has two distinct subsystems. The retriever takes a user query, searches a knowledge base (vector store, keyword index, hybrid search, or some combination), and returns a set of candidate passages. The generator takes those passages plus the original query and produces an answer.

These two subsystems interact, but their failure modes are independent. A retriever with perfect recall feeding a sloppy generator still produces garbage. A brilliant generator receiving irrelevant context hallucinates. You cannot derive end-to-end quality from either metric alone, and you cannot debug a bad answer without knowing which subsystem caused it.

The practical implication: a RAG evaluation pipeline must measure retrieval quality, generation quality, and end-to-end quality, each with separate metrics. The next two sections cover what those metrics are.

Evaluating Retrieval: Precision@k, Recall@k, MRR, NDCG

Retrieval evaluation answers one question: when we search for relevant documents, do the relevant ones appear near the top of the ranked list? Four metrics dominate the literature, and they serve different purposes.

Precision@k and Recall@k

Precision@k answers: of the top k documents returned, how many are actually relevant? If you return 10 chunks and 4 are relevant, precision@10 is 0.4.

Recall@k answers: of all the relevant documents that exist in the corpus, how many appear in the top k? If the corpus has 20 documents that are truly relevant to the query and the retriever surfaces 12 of them in the top 10 (it can surface at most 10, so it catches 10), recall@10 is 0.5. More realistically, recall@k measures whether the retriever finds what is there.

In practice, we reach for recall@k first. A RAG generator can usually ignore irrelevant chunks (if it is well-behaved), but it cannot invent information the retriever did not surface. Recall below 0.7 at k=10 is a red flag for most production systems. Precision@k matters more when the generator is sensitive to noise — some smaller models degrade noticeably when too many irrelevant chunks appear in the context window.

Mean Reciprocal Rank (MRR)

MRR measures the rank position of the first relevant document. If the first relevant document is at position 1, the reciprocal rank is 1. At position 3, it is 1/3. MRR is the mean of these values across all queries.

MRR is useful when a single relevant passage is enough to answer the question. For a FAQ bot or a lookup-style RAG system, MRR above 0.8 at k=10 is strong. Below 0.6 suggests the retriever is consistently missing the most obvious match.

Normalized Discounted Cumulative Gain (NDCG)

NDCG is the most sophisticated metric of the four. It handles two realities the simpler metrics ignore: some documents are more relevant than others (graded relevance), and a relevant document at position 5 should count for less than one at position 1.

NDCG works by computing the cumulative gain at each rank position, discounting by the log of the rank, and normalizing by the ideal ordering. A score of 1.0 means the ranking is perfect. For a well-tuned RAG pipeline with a decent embedding model (text-embedding-3-large, Cohere Embed v4, or a fine-tuned sentence-transformer), NDCG@10 should be above 0.85. Below 0.7 means the ranking needs work — try hybrid search (dense + sparse), reranking, or chunking adjustments.

Putting Them Together

No single metric is sufficient. We use this combination:

• Recall@k to catch cases where the retriever missed relevant documents entirely (the most damaging failure).
• Precision@k to measure the noise floor the generator has to deal with.
• MRR or NDCG (pick one, not both in dashboards) to track ranking quality improvements over time.

A practical threshold set for a production RAG system with a capable generator (GPT-5.5 Instant, Claude Fable 5, Gemini 3.1 Pro):

Evaluating Generation: Faithfulness, Relevance, Completeness

Retrieval metrics tell us whether the model had the right information. Generation metrics tell us whether it used it correctly. Three dimensions capture the vast majority of generation failures in RAG.

Faithfulness (a.k.a. Groundedness)

Faithfulness measures whether every claim in the generated answer can be traced back to the retrieved context. This is the antidote to hallucination. A high-faithfulness answer is entirely grounded. A low-faithfulness answer invents facts, mixes context details incorrectly, or contradicts the retrieved passages.

The canonical way to measure faithfulness is claim decomposition: split the generated answer into individual factual claims, then check each claim against the retrieved context. This can be done by an LLM judge or by a dedicated NLI (natural language inference) model. The faithfulness score is the fraction of claims that are supported.

A faithfulness score below 0.8 is concerning. Below 0.6 means the generator is regularly ignoring or overriding the provided context, which defeats the purpose of RAG entirely.

Common failure patterns caught by faithfulness:

• Context ignored: The model answers from its own knowledge even though relevant context was provided. This happens when the prompt does not sufficiently emphasize the context, or the model’s training dominates the retrieval signal.
• Hallucinated entities: The answer mentions specific names, dates, or numbers that do not appear in the context.
• Conflation: The answer combines details from multiple chunks incorrectly (e.g., assigning one person’s accomplishment to another person mentioned in a different chunk).

Answer Relevance

Answer relevance measures whether the generated response actually addresses the user’s query. It is possible to have a faithful answer that does not answer the question — for example, the model recites facts from the retrieved context that are technically true but irrelevant to what the user asked.

Relevance is typically measured by having an LLM judge generate candidate questions from the answer, then computing the cosine similarity between those questions and the original query. If the generated questions semantically match the original query, the answer is on-topic. If they diverge, the answer missed the mark.

A relevance score above 0.9 indicates the answer directly addresses the query. Below 0.7 means the system needs work — the retriever may be surfacing tangentially related content, or the generator may be drifting into generic responses.

Completeness (a.k.a. Answer Coverage)

Completeness asks: did the answer cover everything in the context that was relevant to the question? This is the most frequently overlooked dimension. An answer can be faithful (all claims are grounded) and relevant (it addresses the query) but still miss important information that was present in the retrieved chunks.

Consider a query: “What are the side effects and contraindications of drug X?” The retriever surfaces documents covering both. The generator faithfully describes the side effects but never mentions contraindications. Faithfulness passes. Relevance partially passes (it answered about side effects). Completeness catches the gap.

Completeness is measured by comparing the answer against the retrieved context and a ground-truth “expected coverage” for the query. The score is the fraction of expected information points that appear in the answer.

A completeness score below 0.7 often points to the retriever returning only partial coverage, which the generator then faithfully reproduces. In that case, the root cause is retrieval, not generation — even though the symptom looks like a generation problem.

Generation Metric Summary

Building the Evaluation Dataset

You cannot evaluate a RAG system without a labeled evaluation dataset. The dataset needs three components for each query:

1. The query — a realistic user question drawn from production logs or domain experts.
2. Ground-truth relevant documents — the set of documents (by ID) that the retriever should return for this query. Every retrieval metric depends on this annotation.
3. Expected answer coverage — a list of the key information points the answer should cover. This is needed for completeness evaluation.

How to Build It

Start with production queries. If your RAG system is already running, extract the real queries users are sending. If it is not yet live, work with domain experts to write 100-200 representative questions. Cover edge cases: ambiguous queries, multi-part questions, questions that require documents from different sources.

Annotate relevant documents. For each query, a human annotator (or a domain expert, or a trusted subject-matter expert) identifies which documents in the corpus are relevant. This is the most expensive part of building an eval set, but it is non-negotiable. Without ground-truth relevance labels, recall@k and NDCG are unmeasurable.

Write expected coverage. For each query, list the factual points the answer should contain. This does not need to be exhaustive — three to five bullet points per query is usually sufficient to catch completeness failures.

Minimum viable size. A robust eval set is 100-200 queries. With fewer than 50, the confidence intervals on your metrics are too wide to trust. With more than 500, you hit diminishing returns on annotation effort. The sweet spot is around 150 queries for most teams.

Augment with synthetic data. If annotation resources are tight, use a strong LLM (GPT-5.5 Instant, Claude Fable 5) to generate candidate queries from your documents, then have a human review and correct them. This is much faster than writing queries from scratch and produces reasonable eval sets, as long as every synthetically generated query is reviewed — unreviewed synthetic queries introduce systematic biases (models generate questions they can already answer well).

RAG Failure Modes and Which Metrics Catch Them

Every RAG failure has a signature in the evaluation metrics. Here is the diagnostic mapping:

The last row is subtle. A model that answers correctly from its own knowledge may score well on end-to-end metrics, but the RAG system is not doing its job. You detect this by measuring context utilization: of the claims in the answer, what fraction rely on the retrieved context versus the model’s parametric knowledge. This requires an extra evaluation step where you check whether each claim appears in the retrieved chunks.

Evaluation Frameworks: RAGAS, TruLens, DeepEval

Building all these metrics from scratch is possible but not advisable. Several frameworks implement them out of the box.

RAGAS

RAGAS (Retrieval Augmented Generation Assessment) is the most widely used open-source framework for RAG evaluation. It implements faithfulness, answer relevance, and context precision/relevancy, plus a handful of composite scores.

Strengths: comprehensive metric coverage, active community, integrates with LangChain and LlamaIndex. Weaknesses: the default LLM judge can be inconsistent — you get better results by configuring a strong judge model (Claude Fable 5 or GPT-5.5 Instant) rather than the default. RAGAS also requires a well-structured eval dataset with ground-truth contexts.

TruLens

TruLens provides a feedback-functions approach with three core metrics: groundedness (faithfulness), answer relevance, and context relevance (does the retrieved context match the query). It includes a dashboard for tracking metrics over time and comparing different pipeline versions.

Strengths: excellent observability story — version tracking, feedback logging, and an interactive UI. Weaknesses: heavier dependency footprint, and the dashboard requires a running database (but works fine with SQLite for local use).

DeepEval

DeepEval offers a unit-testing metaphor for LLM evaluation. It implements RAG metrics (faithfulness, relevancy, hallucination, context recall/precision) alongside non-RAG metrics (G-Eval, summarization, toxicity). It works with pytest, which makes it attractive for teams that want eval-as-test.

Strengths: pytest integration, clean API, good documentation. Weaknesses: less mature than RAGAS, smaller community.

Which One to Pick

All three support custom judge models, custom metrics, and async evaluation. You cannot go wrong starting with any of them — the important thing is to start.

Building a RAG Eval Pipeline

A production eval pipeline runs on every significant change: new embedding model, new chunking strategy, new prompt template, new generator model. Here is a practical pipeline structure.

Step 1: Run retrieval. For each query in your eval set, run the retriever and record the top-k results (k=10 is a good default). Store document IDs and similarity scores.

Step 2: Compute retrieval metrics. Calculate precision@k, recall@k, MRR, and NDCG against the ground-truth relevant documents. Log all scores and per-query breakdowns.

Step 3: Run generation. For each query, feed the retrieved chunks plus the query to the generator. Store the generated answer.

Step 4: Compute generation metrics. Run faithfulness, answer relevance, and completeness evaluations. Use an LLM judge that is different from (or at least as capable as) the generator model. If your generator is Claude Fable 5, use GPT-5.5 Instant as the judge and vice versa, to avoid systematic biases.

Step 5: Compute end-to-end metrics. If you have ground-truth ideal answers for your eval queries, compute lexical metrics (ROUGE-L, BLEU) or semantic similarity (embedding cosine similarity) between the generated answer and the ideal answer. These are noisy but useful as a sanity check.

Step 6: Flag regressions. Compare against the previous evaluation run. Flag any metric that drops beyond the action threshold. Per-query breakdowns tell you exactly which queries regressed — investigate those individually.

Practical Thresholds for a CI Pipeline

Build your CI gates around the distribution of per-query scores, not just the mean:

Averages can hide a lot. A system with 95 percent of queries scoring 0.9 and 5 percent scoring 0.0 has a mean of 0.855 — looks fine until the 5 percent are your most important queries. Always enforce floor thresholds.

Debugging with Evaluation Results

When your RAG system produces bad answers, the evaluation metrics tell you where to look first. Here is the debug protocol.

Symptom: User reports the answer is wrong or makes things up. Look at faithfulness first. If it is low, the generator is hallucinating. Check whether the retrieved context actually contained the correct information (look at recall@k for that query). If recall is fine, the problem is generation — tighten your prompt, try a different model, or add a factual consistency check. If recall is low, the problem is retrieval — the generator could not ground its answer because the information was not there.

Symptom: User reports the answer is too short or skips important details. Look at completeness. If completeness is low but recall is fine, the generator is truncating or the prompt is not instructing it to cover all relevant information. If completeness is low and recall is also low, the retriever failed to surface all the relevant documents.

Symptom: User reports the answer is irrelevant to their question. Look at answer relevance and precision@k. Low precision@k means the retriever polluted the context with irrelevant documents. Low answer relevance means the generator did not answer the question even if it had some relevant context. Both metrics low points to retrieval as the culprit.

Symptom: User reports the answer feels generic, like the model is not using our data. This is the hardest one to catch with standard metrics because the answer might be factually correct. Compute context utilization — the fraction of claims that are grounded in the retrieved context versus the model’s parametric knowledge. Low context utilization means your prompt or system instructions are not steering the model to rely on the provided context.

The Debug Decision Tree

Answer quality problem?
  |
  +-- Faithfulness < 0.8?
  |     +-- Recall@k < 0.7?  --> Fix retriever (chunking, embeddings, hybrid search)
  |     +-- Recall@k >= 0.7? --> Fix generator (prompt, model choice, add guardrails)
  |
  +-- Completeness < 0.7?
  |     +-- Recall@k < 0.7?    --> Fix retriever
  |     +-- Recall@k >= 0.7?   --> Fix generator prompt to encourage thoroughness
  |
  +-- Answer relevance < 0.9?
  |     +-- Precision@k < 0.6? --> Fix retriever (reranker, similarity threshold)
  |     +-- Precision@k >= 0.6? --> Fix generator prompt to stay on-topic
  |
  +-- All metrics nominal but answers feel generic?
        --> Compute context utilization
              +-- Low --> Fix prompt to force context reliance
              +-- High --> Accept — system is working correctly

This tree will get you to the right debugging target in about 80 percent of cases. The remaining 20 percent require examining specific queries end to end — which the per-query evaluation logs make straightforward.

Practical Takeaways

Evaluate both subsystems independently. Retrieval metrics and generation metrics catch different failures. Running only end-to-end evaluation tells you something is wrong but not what.

Invest in the evaluation dataset. Your evaluation is only as good as your ground-truth labels. Spend the time to get 150 annotated queries with document relevance labels and expected coverage. This is the highest-leverage investment you can make in RAG quality.

Set floor thresholds, not just averages. A single failing query among a hundred good ones can destroy user trust. Track the minimum score, not just the mean.

Use different models for generation and evaluation. The strongest evaluation setup uses one model as the generator and a different (but equally capable) model as the judge. This avoids the blind spots that come from using the same architecture for both tasks.

Run evaluation on every change. Chunk size adjustments, embedding model swaps, prompt tweaks, new LLM versions — every change needs a full eval run before shipping. Automate it in CI with the thresholds above.

Start simple, iterate. Begin with RAGAS and a single retrieval metric plus faithfulness. Add completeness, NDCG, and context utilization as your maturity grows. A working evaluation pipeline with three metrics beats a perfect one that never ships.