Detecting Hallucinations in Production

Your application’s logs look clean. No crashes, no timeouts, no error codes. But somewhere in that stream of plausible-sounding responses, the model just told a customer that their account was charged $240 when it was $24 — a decimal point’s worth of hallucination that could trigger a support ticket, a chargeback, or worse. Hallucination detection is fundamentally different from every other monitoring category we’ve built in production systems. Monitoring for crashes, latency, or error codes follows a familiar pattern: you define a threshold, you alert when it’s breached, and someone wakes up to fix it. Hallucinations invert the problem. Instead of looking for something breaking loudly, we’re hunting for outputs that look perfectly reasonable but are simply wrong in ways that matter. The error signal isn’t an exception trace — it’s a flawlessly constructed sentence containing a falsehood. That changes everything about how we detect, measure, and respond.

Why Hallucination Detection Is Different

Traditional monitoring works because errors emit signals. A 500 status code, a segmentation fault, a null pointer — each produces a detectable artifact that propagates upward through your stack. The failure mode is loud, and it’s consistent: the same input will crash the same endpoint every time.

Hallucinations produce none of these signals. The model returns a 200 OK with perfectly formatted JSON and natural-sounding prose. Your pager stays silent. The only person who notices the error is the end user, and they notice it not because the system broke but because they happen to know something the model got wrong.

This makes hallucination detection a semantic monitoring problem rather than a syntactic one. We’re not looking for malformed data; we’re looking for well-formed data that encodes a false proposition. That requires an entirely different toolchain — one built on natural language understanding, reference-based verification, and probabilistic uncertainty estimation rather than regex patterns and error budgets.

The Hallucination Severity Spectrum

Not all hallucinations are created equal, and treating them as a binary (hallucinated vs. not hallucinated) leads to either overwhelming noise or dangerous false negatives. We find it useful to categorize hallucinations along a severity spectrum.

The key insight here is that most production hallucinations are in the middle two buckets. They’re not obviously absurd (the model doesn’t say the sky is green), and they’re not harmless enough to ignore. They’re the plausible errors that slip past users and reviewers alike.

Detection Techniques Compared

There is no single “hallucination detector” you can drop into your stack. Instead, the field has converged on several distinct approaches, each with different strengths, weaknesses, cost profiles, and latency characteristics.

Self-Check Prompting

The simplest approach: ask the model to verify its own output. After generating a response, you prompt the same model (or a smaller one) to check each claim against its training data or against context you provide.

System: You are a fact-checker. Review the following response. For each factual claim,
state whether it is supported by the provided context. List any unsupported claims.

Strengths: Zero additional infrastructure, works with any model, easy to implement in a few hours.

Weaknesses: The model checking itself inherits the same knowledge gaps and blind spots that produced the hallucination in the first place. A model that confidently asserts a false date will often confidently confirm that same false date when asked to check it. Research consistently shows self-check recall sits around 60-70 percent on factual consistency tasks.

Latency: Adds a full generation cycle — roughly 1-2x the original response time.

Cost: Approximately doubles token usage for response generation, though using a smaller “checker” model reduces this.

NLI-Based Verification

Natural Language Inference models classify the relationship between a “premise” (your source document or knowledge base) and a “hypothesis” (each claim in the model’s output) as entailment, contradiction, or neutral. A claim classified as contradiction or neutral is flagged as potentially hallucinated.

Strengths: Strong theoretical foundation, interpretable classifications, well-studied task with established benchmarks. Models like DeBERTa and ModernBERT-based NLI classifiers are widely available and can be fine-tuned on domain-specific data.

Weaknesses: NLI models struggle with complex multi-claim sentences, implicit claims, and claims that require world knowledge beyond the provided premise. They also require you to split model outputs into atomic claims, which is itself a nontrivial NLP task.

Latency: 100-500ms per claim when using a dedicated NLI model, assuming you can run it on GPU. Claim splitting adds overhead.

Cost: Low for inference — a small transformer model handles this efficiently. Training or fine-tuning adds upfront cost.

Cross-Verification Against Source Documents

For retrieval-augmented generation (RAG) systems, this is the most natural approach. Each claim in the model’s output is checked against the retrieved documents that were provided as context. If a claim cannot be attributed to any source document, it’s a candidate hallucination.

The practical implementation involves:

1. Claim extraction: Split the model output into atomic, verifiable claims.
2. Attribution scoring: For each claim, compute the semantic similarity against each source document chunk.
3. Thresholding: Claims below a similarity threshold are flagged.

Strengths: Directly addresses the most common hallucination mechanism in RAG systems (models ignoring or misreading retrieved context). Aligns with how users intuitively fact-check.

Weaknesses: Only works when you have source documents to check against. Doesn’t catch hallucinations based on the model’s parametric knowledge. Requires careful tuning of the similarity threshold and claim extraction quality.

Latency: 200-800ms depending on document count and chunk size.

Uncertainty Estimation via Token Probabilities

Models produce a probability distribution for each generated token. Low-probability tokens — or, more usefully, sequences of low-probability tokens — can indicate uncertainty. The most common metrics are:

• Mean token log-probability: Average confidence across all generated tokens.
• Semantic entropy: Clusters model outputs by meaning and measures entropy across clusters. Repeated generations that say different things indicate uncertainty.
• Token-level perplexity spikes: Sudden drops in probability on key factual tokens (dates, names, numbers).

Strengths: No external model or infrastructure needed. Can run at inference time with near-zero additional latency if you’re already logging token probabilities.

Weaknesses: Correlated with but not causal for hallucinations. Models can be confidently wrong — a token can have probability 0.999 and still be a hallucination. Semantic entropy requires multiple model generations, which is expensive.

Latency: Minimal if you’re logging probabilities during generation. Semantic entropy requires 3-5 generations per input (roughly 3-5x latency).

Reference-Based Factuality Scoring

This category includes purpose-built evaluation frameworks that combine several of the above techniques into scored metrics. These are the most production-ready options available today.

RAGAS Faithfulness: Designed specifically for RAG pipelines, RAGAS decomposes the model output into atomic claims and checks each one against the retrieved context using an NLI-style approach. It produces a 0-1 faithfulness score per response. Implementation is straightforward via the ragas Python library, and it integrates with LangChain, LlamaIndex, and most observability platforms.

TruLens: Offers a “groundedness” feedback function that measures whether each claim in the response is supported by the context. TruLens runs as a sidecar evaluation service and can be called asynchronously after the response is delivered, making it suitable for production flag-and-review workflows.

DeepEval: Provides a comprehensive suite of metrics including hallucination_metric, faithfulness_metric, and contextual_recall. DeepEval’s hallucination detector uses a multi-step pipeline: claim extraction, NLI verification, and cross-document consistency checking. It reports not just a score but the specific claims that failed verification.

Production Patterns That Work

The techniques above are ingredients, not recipes. How you combine them depends on your latency budget, your risk tolerance, and your infrastructure. We’ve seen three patterns emerge in production deployments.

Guard Models That Screen Outputs

Place a smaller, cheaper model between your primary model and the user. The guard model receives the primary model’s output and either passes it through, flags it for review, or blocks it entirely. This is the highest-confidence pattern because it catches hallucinations before they reach the user.

A typical guard model pipeline:

User Query → Primary LLM → Guard Model → [Pass | Flag | Block] → User

The guard model runs a checklist: Does the response contain numerical claims? If yes, can each number be verified against the context? Are there citations, and do they correspond to real documents? Does the tone express certainty about uncertain facts?

When to use: High-stakes applications (medical, legal, financial) where the cost of a hallucination is severe. Systems with moderate QPS where adding a guard model doesn’t bottleneck throughput.

Trade-off: Doubles inference cost per request and adds latency. But it’s the only pattern that prevents hallucinations from reaching users rather than detecting them after the fact.

Factuality Scoring at Output Time

Score every response asynchronously and route low-scoring responses to a review queue. This pattern accepts that some hallucinations will reach users but catches them quickly enough to mitigate damage.

User Query → Primary LLM → User receives response (immediately)
                                   ↘ Factuality scorer (async)
                                       ↓ Low score?
                                       → Yes: Alert team, offer correction
                                       → No: Log and move on

When to use: Customer-facing chat, support bots, content generation tools. Any system where sub-second response time is critical but some risk is tolerable.

Trade-off: Users see the hallucination before you catch it. But you preserve low latency, and you build a dataset of flagged responses that improves your evaluators over time.

Reference-Based Verification for RAG

For RAG systems specifically, verify every response against its retrieved context before delivery. This is a specialization of the guard model pattern that exploits the fact that you already have source documents to check against.

The implementation:

1. After the model generates a response, split it into claims.
2. For each claim, retrieve the top-K source chunks that were in the model’s context window.
3. Run an NLI classifier to check if each claim is entailed by its corresponding source chunk.
4. If any claim fails, regenerate with a stronger system prompt emphasizing adherence to context, or block the response and surface the source documents directly.

When to use: Any production RAG system — internal knowledge base Q&A, customer support over documentation, code generation from internal libraries.

Trade-off: Adds 500ms-2s of latency per response but dramatically reduces the most common class of RAG hallucination: the model making things up that contradict or extend beyond the retrieved documents.

Building a Hallucination Monitoring Dashboard

Detection without visibility is just noise. If you’re scoring every response for factuality, you need a dashboard that surfaces actionable patterns rather than raw scores. Here’s what we’ve found essential.

Per-response score distribution: A histogram of factuality scores over time. Watch for shifts — a sudden drop in average scores often correlates with a data drift issue or a model update.

False positive rate tracking: Log every response your system flags and, when possible, collect user feedback on whether the flag was justified. A high false positive rate destroys trust in the monitoring system and leads to alert fatigue.

Score breakdown by domain or topic: Models hallucinate more in some areas than others. If your financial responses average 0.92 faithfulness but your medical responses average 0.78, that’s a signal to invest in domain-specific retrieval pipelines or system prompts.

Latency waterfall: Show the time spent in each verification step — claim extraction, NLI inference, attribution scoring. This helps you decide where to optimize.

Verification coverage: Track what percentage of claims your system can actually verify against source documents. If coverage drops below 90 percent, you’re likely missing hallucinations because the claims are about topics not present in your knowledge base.

Cost and Latency Trade-Offs

Every detection technique has a cost, and not every application can afford all of them. Here’s our rough rule of thumb.

These are rough numbers, and the actual gains depend heavily on your domain, your base model, and the quality of your retrieval pipeline. But the pattern is clear: you get diminishing returns past a certain point. Token probabilities are nearly free and catch the low-hanging fruit. Full scoring pipelines capture more but at 10-100x the cost.

The Honest Truth: Defense in Depth

No single method catches everything. The research literature and production experience both point to the same conclusion: hallucination detection is a defense-in-depth problem, not a silver-bullet problem.

Here’s what a defense-in-depth strategy looks like in practice:

Layer 1 — Input guardrails: Prevent the model from being prompted into hallucination-prone territory. If a user asks about a topic outside your knowledge base, redirect or set expectations before the model generates anything.

Layer 2 — Retrieval quality: In RAG systems, the most common root cause of hallucinations is bad retrieval, not bad generation. If the model doesn’t have the right documents, it will fabricate. Invest in retrieval quality before you invest in detection.

Layer 3 — Output-time detection: Apply one or more of the techniques above depending on your risk profile and latency budget. Start with self-check or token probabilities for broad coverage, then layer on NLI or reference-based scoring for high-risk outputs.

Layer 4 — Post-hoc monitoring: Log everything — scores, claims, verification results — and surface patterns in a dashboard. Use this data to improve your prompts, your retrieval, and your evaluators over time.

Layer 5 — Human review loop: For high-stakes applications, route flagged responses to human reviewers. Use their judgments as ground truth data to continuously improve your automated detectors.

Practical Takeaways

If you’re building a hallucination detection system today, here’s where we’d start:

Start with RAGAS or DeepEval for offline evaluation. Before you build real-time detection, understand your baseline. Run batch evaluations on your test set and identify the most common hallucination categories in your domain.

Log token probabilities immediately. It costs nothing, adds zero latency, and gives you a weak but always-available signal. You’ll be surprised how often low-confidence tokens correlate with the responses your users complain about.

Pick one production technique and get it right. Don’t try to deploy all five methods at once. Start with cross-document verification for RAG systems or self-check prompting for pure LLM applications. Build the dashboard, tune the thresholds, learn the failure modes. Then add a second layer.

Invest in claim extraction. Every verification technique depends on splitting model outputs into atomic, verifiable claims. If your claim extraction is poor, no downstream detector will save you. Test and iterate on this step specifically.

Accept the gap. No production system catches every hallucination. The goal is not 100 percent detection — it’s catching the ones that matter most while maintaining a low false positive rate that keeps your team trusting the system. That balance defines a successful deployment.

Hallucination detection is the most challenging monitoring problem most engineering teams will face in the next few years, precisely because the error signal is semantic rather than mechanical. But with the right techniques, the right dashboards, and a sober understanding of what’s possible, you can build systems that catch the worst hallucinations without drowning in noise. The models will keep getting better, but the monitoring problem — distinguishing the plausible false from the plausible true — will be with us for a long time.