Microsecond word-overlap check beats DeBERTa NLI at catching RAG hallucinations

The result

A microsecond-scale word-overlap check outperformed a neural entailment model at spotting hallucinated responses from retrieval-augmented generation systems, according to a benchmark write-up published on dev.to by Guillermo Llopis of ProtForge. The lexical detector, which scores how much of an answer's vocabulary never appears in its retrieved source, reached an AUROC of 0.778 on a RAGTruth test sample. A SummaC-style pipeline built on the cross-encoder/nli-deberta-v3-small model scored 0.716 on the same data, while costing roughly 7.3 seconds per response on a loaded four-thread CPU versus microseconds for the lexical check.

How the comparison was run

The evaluation used RAGTruth, a hallucination corpus from Niu et al. (ACL 2024) with about 18,000 responses from six LLMs to retrieval-grounded prompts, spanning question answering, summarisation and data-to-text. Human annotators marked hallucinated spans; a response counted as hallucinated if it had at least one.

Three detectors were compared, each returning a score between 0 and 1 for a source-response pair: a lexical check measuring the share of content tokens (lowercased, stopwords and punctuation removed, numbers kept) absent from the source; a numeric check measuring the share of response numbers missing from the source after normalising formats such as 1,000 and 1k; and an NLI check scoring each response sentence against its three closest source sentences. A fourth ensemble detector combined the three scores with logistic regression.

Thresholds and ensemble weights were fitted on 299 stratified training responses and frozen. Headline numbers come from a stratified 900-response test subsample (300 per task) with a 35.3% hallucination base rate. Because of the NLI model's CPU cost, it was scored only on the subsample; the lexical detector also ran on all 2,700 test responses, where it scored 0.785, so the subsample was not flattering it.

What the numbers show

The lexical detector achieved AUROC 0.778 (95% CI 0.746-0.806), precision 0.476, recall 0.858 and F1 0.612. NLI followed at 0.716 (0.681-0.747) with the same precision but slightly lower recall (0.836) and F1 (0.607). The numeric detector managed only 0.535, and its best threshold was 0.0, effectively flagging every response containing a number absent from the source. The ensemble reached 0.766, failing to beat lexical overall and leading only on question answering.

The author frames the headline gap cautiously: the confidence intervals barely overlap, so the fair reading is that lexical matched or beat NLI in this specific setup rather than proving a general rule. The picture underneath varies. NLI fell to chance (0.496) on data-to-text, where sources are flattened JSON and 66% of responses are hallucinated. Per generator model, NLI led on Mistral 7B Instruct (0.812 against 0.688 for lexical) but trailed on GPT-4 (0.689 against 0.808), though with 150 responses per model those intervals are wide. The pooled 0.78 also exceeds every per-task figure, partly because detectors latch onto which task a response belongs to, given how much base rates differ across tasks.

Why the lexical check holds up

The author's explanation, explicitly flagged as unproven, is that most RAGTruth hallucinations add new material such as a name, a claim or a plausible detail the source never mentions, which a test for unseen vocabulary catches for free. The NLI model faced a harder job in a constrained setup: it saw only the top three source sentences per response sentence, and its probabilities saturate near 0 and 1, pushing the tuned threshold to 0.9987.

Failure modes also differ. NLI caught a clean-sounding fabrication about a racing circuit that lexical scored low, while lexical caught plausible boilerplate that NLI passed, which motivates combining the two, undercut by the ensemble's failure to improve on lexical alone.

Caveats

Precision is poor across the board. At the threshold tuned for F1, about half of correct responses also get flagged, per the author's own arithmetic from the table, so a flag is close to a coin flip. Predicting "hallucinated" for everything scores F1 0.522, meaning 0.612 is a modest gain. Thresholds fitted on 299 responses will not transfer to other data. No LLM-as-judge was run, so the comparison says nothing about that alternative. Scope is English-only, three tasks, 2023-era generator models, response-level labels and no fine-tuning. The post also discloses it was drafted by an AI assistant from the repository's results files and reviewed by the author.

Why it matters

For teams building RAG guardrails, the finding suggests starting with an essentially free lexical score before paying for neural scorers or LLM judges, and treating it as a ranking signal for what gets human or model review rather than an automatic blocker, since blocking flagged answers would discard many good ones. Because detector rankings shifted with task and generator, the practical advice is to label a few hundred responses from your own traffic and measure. The code is MIT-licensed in the groundcheck repository on GitHub under protforge-ai, with a fuller write-up on ProtForge's site.