AI hallucination detection that classifies every claim and corrects the real errors

A grounding score tells you a response strayed from its source. It rarely tells you what to do about it. “42% grounded” names a problem and hands it straight back to you, with the real work, finding the bad claims and fixing them, still to do.

A more useful check carries that work further:

• reads a response one claim at a time,
• says why each unsupported claim failed,
• corrects the ones a rewrite can honestly fix, and
• names the ones it cannot.

The run below does exactly that, start to finish, on a short company bio.

The source and the response

Here is the source, a short paragraph about a fictional robotics company, the kind a knowledge base might hold. Read the two sentences about when it was founded closely, they will matter later.

Here is a response that summarises the company. It reads like a polished “about us” blurb, and several of its most confident lines are not in the source at all.

Read without the source, nothing stands out. That is the failure mode in a sentence. The invented parts are the most fluent parts.

The check scores 0 out of 100, with a reason for every claim

The response was scored against the source with a faithfulness rubric. A rubric is the written standard the response is graded against, a short list of named criteria, each with a weight that sets how much it counts toward the final score. A faithfulness rubric is built so a fluent answer that invents facts scores low rather than high, because its criteria reward grounding in the source, not polish. Here is the one used for this run.

No model generated anything. The judge read the exact response against the exact source, graded it on those four criteria, and returned its reasoning alongside the number. It came back 0 out of 100, and it pointed at each planted error:

• “over 500 people” conflicts with the source, which states roughly 180.
• “LIDAR-based obstacle avoidance” restates what the source says (LIDAR and wheel odometry, for navigation).
• “industry-leading battery life” inflates a plain fact (up to 9 hours) into a claim the source never makes.
• the Forbes award appears nowhere in the source.

A claim can fail for four reasons, and only two can be fixed

This is the part a plain score skips. When a claim is not backed by the source, there is more than one reason why, and the reason decides whether a rewrite can fix it.

• Contradicted. The source states something that conflicts with the claim (the 500 employees). Fixable: correct it to match the source.
• Underspecified. The source touches the topic but supports less than the claim states (“industry-leading battery life”). Fixable: pull the claim back to what the source actually supports.
• Not in source. The source is silent, so there is nothing to ground the claim (the Forbes award). Not fixable by rewriting.
• Ambiguous. The source itself is unclear or self-conflicting, so the claim cannot be confirmed from it. Not fixable by rewriting.

The first two are errors in the response, and a rewrite corrects them. The last two are not, and inventing grounding for them would be the very hallucination the check exists to catch.

The fix corrects the errors and drops what cannot be grounded

The repair made three moves:

• corrected the contradiction to match the source,
• pulled the inflated claims back to what the source supports, and
• dropped the Forbes award, because there is no honest way to ground a claim the source never makes.

It removed that last claim rather than soften it or invent a citation for it.

With the contradictions corrected, the overstatements pulled back, and the ungroundable claim removed, every remaining claim traces to the source. The repaired response scored 100 out of 100, every criterion clean.

That is the full loop in one run, a response that started as confident fiction, scored, diagnosed claim by claim, and rewritten into something faithful. The whole sequence is a single recipe you hand your agent, the ground-check-and-fix recipe, which runs the check and the budgeted repair against a source you supply.

The most useful finding pointed at the source

The source also stated when the company was founded, and this is the case worth the most attention. Checked against the full source, the founding year could not be grounded either way, because the source gives two different years in two different places. The source disagrees with itself.

No rewrite of the response can honestly fix that. The problem is the source, not the answer. So the right outcome is a flag, not a repair:

• The claim cannot be grounded, because the knowledge base contradicts itself.
• The repaired response leaves the founding year out and points back at the contradiction to resolve.

That is a grounding check earning its keep. It cleaned up the answer and surfaced a flaw in the source the answer was drawing from, the kind of flaw that would quietly affect every future answer built on that document.

A passing score means read it, not ship it

One honest limit, because it is part of the method. A repair loop that chases a faithfulness score can pass the wrong way, by qualifying every claim into vagueness or quietly dropping the hard ones, and return a clean number that is technically grounded and weaker than where it started. The score is a proxy, and a proxy can be satisfied without the goal being met.

So the loop ends by showing the repaired text and the list of what it changed or dropped, and asks for a human read before anything goes out. A high grounding score means the response clears the bar. It does not replace the final look.

Make it fit your pipeline

The run above is one recipe, and a recipe is a plain-language instruction set you hand your agent, not a fixed black box. Everything about it is yours to adjust:

• Swap the rubric for your own criteria and weights, so “faithful” means what it means for your domain.
• Set the target and the budget, every claim grounded or a score to clear, with your own ceilings on rounds and cost.
• Point it at inline text or at a connected knowledge base, whichever your pipeline uses.
• Tighten where the repair is allowed to act, or where it should only flag and leave the call to you.

Start from the shipped version, change what you need, and your agent runs the version that fits how you work. The grounding check becomes a step in your pipeline, shaped to it, rather than a separate tool you visit.

The short version

A grounding check that stops at a number leaves the real work to you. A check that carries it further does six things:

1. Grades the response against its source one claim at a time, not as a single number.
2. Names why each unsupported claim failed (contradicted, underspecified, not in source, or ambiguous).
3. Corrects the real errors against the source.
4. Flags the claims no rewrite can honestly fix, instead of inventing grounding for them.
5. Surfaces the claim that reveals a flaw in the source itself, not the response.
6. Ends with the repaired text and a prompt to read it, because a passing score clears the bar rather than guaranteeing the result.

Detecting the made-up claim is the starting point. Knowing which kind of wrong it is, correcting what the source can support, and catching the problem that lives in the source rather than the answer, is what keeps bad facts out of production.