How to stop AI hallucinations

The short answer

You cannot stop AI hallucinations completely. You can make them rare, visible, and traceable, and you can put a number on how rare. Any vendor who promises zero either has not measured their own system or is counting on you not to check.

Four mechanisms do the real work, and they stack in this order. Grounding makes the model read your records before it answers. Citations make every claim point back at the record it came from. Abstention gives the system a way to say it does not have the answer, and a reason to take it. Evals measure all three on your own questions and block a release that makes any of them worse. A vendor with all four can show you numbers. A vendor with one or two is showing you a demo.

Ilayer's own homepage carries the line this page is built on: grounding and citations do not make mistakes impossible, they make them visible and traceable, instead of silent. That is not a disclaimer. It is the product. A wrong answer you can trace back to a bad record is a maintenance ticket someone closes on Tuesday. A wrong answer with nothing behind it is a decision your team makes badly, six months from now, without ever learning why.

Why nobody can sell you zero hallucinations

A model does not store facts, it stores patterns, and an unanswerable question still produces its most plausible continuation. Kalai and colleagues argue the habit is reinforced by how models are scored, since training and evaluation reward guessing over acknowledging uncertainty (Why Language Models Hallucinate1). The mechanics belong to another page: why AI makes things up covers the cause properly. For a buyer, only the consequence matters.

The consequence is that retrieval helps enormously and does not finish the job. Stanford's RegLab benchmarked three commercial legal research tools, all of which retrieve from real source law before answering. Lexis+ AI and Ask Practical Law AI still hallucinated on more than 17 percent of benchmark queries, and Westlaw's AI-Assisted Research on more than 34 percent (Stanford HAI2). Those products have shipped changes since, and the specific scores will age. The shape of the result will not: bolting retrieval onto a model reduces invention rather than removing it, and the marketing around those tools was well ahead of what the measurements supported.

So the question to carry into a vendor call is not whether the system hallucinates. It is whether you will find out when it does.

The four mechanisms, and what each one leaves behind

Each of the four catches something the one before it misses, which is why they are worth buying in order rather than picking a favorite. Grounding narrows what the model can say. Citations expose what it did say. Abstention handles the questions grounding could not serve. Evals tell you whether any of it still works after last Thursday's change. The most common half-built configuration in the market is grounding with no abstain path: an accurate-looking system that has no way to admit a gap, so it fills one.

Grounding: make it read before it answers

Grounding means the system retrieves from your documents and data first, then answers only from what it found. On one production system built for a real-estate investment firm managing about 3.4 billion dollars, that is 29.08 million retrievable chunks drawn from 1,314,631 documents, so a question about a lease clause gets answered from the lease rather than from a general model's impression of what leases usually say.

The failure mode moves rather than disappearing. If the right document is not retrieved, the model answers faithfully from the wrong one and sounds exactly as certain. That is why most of the engineering goes into the retrieval layer, not the model. Whether you should retrieve at all or train a model on your data is a genuine fork with a genuine answer, covered in RAG versus fine-tuning; most business questions want retrieval. Building that layer is what AI agents and grounded RAG is.

Citations: make every claim point at a record

A citation earns its keep only if a human can follow it and a machine already has. On the same system, answers pass a seven-stage verifier that checks each claim against the source it cites before anything is served, and a claim that cannot be traced fails closed instead of shipping with a hedge.

The residual failure here is the citation that is real but does not support the sentence attached to it. That is the most dangerous class of error in the whole category, because it survives a casual spot check. Ask any vendor whether verification confirms the cited passage contains the specific value, or merely that a citation exists.

Abstention: give it a way to say it does not know

An honest "I do not have that" beats a confident wrong answer in every workflow where somebody acts on the output. On the same production system, abstain precision holds at 1.000: every time it declined to answer, declining was correct. Alongside that, zero cross-tenant leaks across both embedding spaces, so one client's records never appeared in another client's answers.

Read the limit of that number, because a vendor quoting it will not. Abstain precision measures whether refusals were justified. It does not measure how often the system should have refused and answered anyway, and it does not measure how often it refuses something it could have handled. Always ask for the refusal rate next to it. A system that abstains on 60 percent of questions can post perfect abstain precision and still be worthless.

Evals: turn "it seemed fine" into a number

An eval suite is a test suite for answers: a set of graded questions with known-correct outputs, run on every change, with thresholds that block a release. The gate on that production system holds zero golden wrong-values and zero determinism violations, meaning the known-answer set stays right and the same question keeps returning the same answer. A change that drops a threshold gets reverted, not argued about.

Method is its own subject, and how to evaluate a RAG system covers building the golden set and reading the scores. At buying time the test is blunter. If a vendor cannot show you a suite and a threshold, then nobody knows whether last week's change made the system worse, including them. That work is LLM evals and reliability, and it is the layer teams skip first and regret longest.

If the budget only stretches to two of the four, take abstention and evals. Grounding and citations without them produce a system that is usually right and never tells you which times. Abstention and evals produce a system that is narrower, occasionally frustrating, and honest about its own edges, which is the version you can put in front of a customer while you widen it. That ordering also survives contact with a legal or compliance review, which the confident-and-unmeasured version does not.

What to demand from any vendor

Take these into the call. Count how many get answered with a number and how many get answered with a paragraph.

1. Show me one answer, then show me the record it came from. Live, on a question you bring, not a recorded demo. If following the citation takes more than one click, your team will not do it.
2. What happens when the answer is not in my data? You want to watch the system decline. A vendor who cannot produce a refusal on request has probably not built an abstain path.
3. What is your abstain precision, and what is your refusal rate? Both, or neither is meaningful. Precision without a refusal rate hides a system that says no to everything.
4. Does verification check the value, or only that a citation exists? This separates a real verifier from a link decorator.
5. Show me the eval suite and the thresholds that block a release. Ask what the last reverted change was. A team with a working gate has a story ready.
6. Who writes the golden answers? They have to be your domain experts, not the vendor's engineers, or the test measures the wrong thing.
7. What is retrieval recall on our questions, measured before we sign? A short retrieval test on your real corpus is cheap and predicts most of the outcome.
8. When it is wrong in production, who finds out and how fast? Logging, monitoring, and a named owner. "The user will tell us" is not an answer.

If most of these come back as prose rather than numbers, you are looking at a demo with a roadmap attached. More of the same shape, aimed wider than hallucination, is in questions to ask an AI development company.

Where this lands in your budget

These four mechanisms are most of the cost of a production AI system, and most of the reason pilots stall. A demo needs a model and a prompt. Production needs retrieval that has been tuned against your corpus, verification, an abstain path, and an eval harness with somebody's name on it. Teams price the first and discover the second, which is the pattern behind why AI pilots never reach production.

Budget for the real thing. What an AI agent costs to build breaks down where the money actually goes, and build versus buy for AI agents covers when an off-the-shelf platform is enough and when the honesty layer has to be yours.

The bar is not a system that never errs. It is a system where every claim has a source, the source is checked before you see it, the system says no when it should, and a number tells you how often all of that holds. Ask for the number. Ask for the failure mode. Ask what gets reverted.