How we cut AI chatbot hallucinations: what actually worked (and what didn't)?

Why Your Chatbot Makes Things Up (It's Not a Bug)

In February 2024, a Canadian tribunal ordered Air Canada to pay damages and honor a bereavement fare policy that its support chatbot had simply invented. The airline argued the chatbot was a "separate legal entity." The tribunal rejected that defense outright.

A year earlier, a US lawyer filed court documents built on ChatGPT output — complete with case names, docket numbers, and quotations that did not exist anywhere. Meta's Galactica research model fabricated citations so confidently it was withdrawn after two days.

These aren't edge cases. They're what happens when a system designed to predict plausible language is asked a question it can't answer — and answers anyway.

Hallucinations are an incentive problem, not a bug. OpenAI's September 2025 research paper argues that standard training and evaluation procedures reward guessing over admitting uncertainty. During pretraining, models see only positive examples of fluent text with no true/false labels, so they learn to "give a guess" — a wrong guess might score points on an accuracy test, while "I don't know" guarantees zero. As OpenAI puts it, if the main scoreboards keep rewarding lucky guesses, models will keep learning to guess.

The data backs this up. In OpenAI's SimpleQA comparison, one model that abstained 52% of the time cut its error rate to 26%, while a guessing model with nearly identical accuracy posted a 75% error rate. Same capability, wildly different defensibility — the only difference was willingness to say "I don't know."

So what does this mean for your chatbot? Three honest expectations:

• Reduced, never eliminated. No source claims full elimination is possible; IBM and Red Hat frame the consensus as "reduced but not eliminated." Some real-world questions are inherently unanswerable.
• Benchmark scores understate your real risk. Top models score under 2% on grounded summarization leaderboards but 16–33%+ on open-domain tests — and enterprise workflows push rates 10 to 40 times higher than benchmark numbers.
• Newer isn't automatically safer. OpenAI's o3 hallucinated on 33% of PersonQA prompts versus 16% for its predecessor o1, because longer reasoning chains generate more claims — each one an opportunity to invent.

The terminology itself is contested — critics like Mary Shaw argue "hallucination" anthropomorphizes software that doesn't know truth from pattern-fit. Whatever you call it, the practical takeaway is the same: a generative model optimizes for statistical plausibility, not correctness.

That's why verification matters more than model choice. It's also why, at SkillzBench, we're building a free directory where agent skills get version-pinned, security-scanned, and benchmarked in a sandbox — with and without the skill — so you can see the Lift before you trust it. The waitlist is live if you want to follow along.

What Actually Worked: Grounding, Structured Prompts, and Rewarding 'I Don't Know'

The three techniques that consistently moved the needle weren't exotic — they came down to controlling what the model can say, how it's asked, and how it's graded. OpenAI, Microsoft, IBM, and Red Hat all land on the same three levers independently.

First: ground everything in curated sources. Retrieval-augmented generation (RAG) forces the model to answer from trusted documents instead of free-associating from its internal parameters. IBM notes that this essentially makes the system cite its answers rather than invent them, while Microsoft's engineering guidance treats RAG as the foundation — with clean data, metadata filtering, and reranking as the quality controls that make grounding actually work. Fine-tuning on curated domain data adds a second layer, teaching the model proper terminology and answer boundaries.

Second: structure the prompt. Microsoft's ICE method — Instructions, Constraints, Escalation — turns vague asks into bounded ones. Instead of "summarize this document," you write: "Using only the retrieved documentation, summarize in 3–5 bullet points. If information is missing, reply 'Insufficient data.'" Two settings matter as much as the wording: repeat key constraints at the beginning of the prompt, and keep temperature between 0.1 and 0.4 for factual work.

Third — and most counterintuitive — reward "I don't know." OpenAI's SimpleQA data makes the case starkly: a model that abstained 52% of the time cut its error rate to 26%, while a model that almost never abstained posted a 75% error rate with nearly identical accuracy. When your own testing penalizes confident errors more than uncertainty, you select for models that know their limits.

• Ground answers in curated, regularly audited sources — not the model's memory
• Use ICE-style prompts with explicit escalation paths and low temperature
• Fine-tune on curated domain examples to teach boundaries, not just vocabulary
• Add guardrails that flag responses containing un-grounded claims
• Score for abstention in your evals, not accuracy alone

One honest caveat: even the best-grounded setup isn't bulletproof. A study of six LLMs found models still struggle to reject false retrieved information — so RAG reduces hallucinations but doesn't eliminate them. The same discipline applies to what you feed agents: skills and instruction packages vary wildly in quality, which is why SkillzBench is building a free directory where every agent skill is version-pinned, security-scanned, and sandbox-tested with and without the skill, so you can see the measured Lift before trusting it. Join the waitlist to get early access when it launches.

What Backfired: Benchmark Scores, Friendly Personas, and RAG as a Silver Bullet

Some of the most intuitive fixes for chatbot hallucinations turn out to be the ones that quietly make things worse. We learned this the hard way — and the research backs it up.

The first trap is picking a model by its leaderboard score. Top frontier models post hallucination rates under 2% on grounded summarization benchmarks, yet the same model generation scores 16–33%+ on open-domain factual tests, and real enterprise workflows push error rates 10 to 40 times higher than benchmark numbers. As one industry analysis puts it, that's like hiring a driver based on their performance on an empty test track — the number is real, but it isn't the road you drive on. The better procurement question: what fraction of your outputs can you actually defend?

The second trap is tuning your chatbot to be maximally agreeable. A friendlier personality feels like better UX, but research summarized on AI hallucination research shows that training a chatbot for a friendlier persona tends to reduce accuracy — agreeable models are simply more inclined to agree with the user. Ethan Mollick's description of ChatGPT as "an eager-to-please intern who sometimes lies to you" captures the risk: politeness and correctness pull in opposite directions.

The third trap is assuming newer reasoning models hallucinate less. OpenAI's own April 2025 system card shows o3 hit a 33% hallucination rate on PersonQA prompts — double the 16% of its predecessor o1. Longer reasoning chains generate more claims, and each claim is another opportunity to invent.

The fourth trap is treating RAG as a silver bullet. Grounding models in retrieved sources is genuinely the foundation — Microsoft, IBM, and Red Hat all converge on it — but a study of six LLMs on the RGB benchmark found models still significantly struggle to reject false retrieved information and integrate it correctly. Models can even contradict the very documents they were just given.

The traps, in short:

• Trusting benchmark scores that collapse 10–40x in production workflows
• Tuning personas for agreeableness at the cost of accuracy
• Assuming newer reasoning models are safer — o3 doubled o1's rate
• Deploying RAG without testing whether the model rejects bad retrieved data
• Running multi-step agents on unverified instructions, where one fabricated premise cascades into destructive actions

That last point matters most for anyone building agents. IBM documents a case where one agent's fabricated metric led a second agent to flag a storage bucket, and a third to delete it — the only copy of financial records under legal hold. This is exactly the problem SkillzBench is building for: a free directory of agent skills where every skill is version-pinned, security-scanned, and benchmarked in a sandbox with and without the skill, so you see the Lift before you trust it. Skills, proven. The waitlist is live now.

Agents Make It Worse: One Hallucination Per Step, Cascading

A single chatbot answer gives a model one chance to hallucinate. An agentic workflow gives it hundreds — and one bad output can become the "fact" every later step builds on.

Microsoft Research measurements show agentic tasks consume roughly 1,000x more tokens than simple chat. Every token is another opportunity to invent, and every intermediate step passes its output — accurate or not — to the next one. The compounding math is unforgiving.

IBM documents exactly how this plays out. In their documented cascade case, Agent A fabricated a storage-bucket inactivity metric, Agent B flagged the bucket for decommissioning, and Agent C deleted it. The bucket held the only copy of financial records under legal hold. One hallucination, three agents, irreversible damage.

The same dynamic explains why newer reasoning models don't automatically help. OpenAI's o3 hallucinated on 33% of PersonQA prompts versus 16% for its predecessor o1 — longer reasoning chains mean more claims, and each claim is a fresh hallucination opportunity. More steps, more surface area for error.

So what does a practical response look like?

• Verify every step's inputs. Treat each agent's output as untrusted until checked against grounding sources — don't let a fabricated premise become the next agent's starting assumption.
• Vet the skills and instructions your agents run with. Remember that published research found roughly 26% of agent skills carry at least one vulnerability, so a skill that "works" can still inject bad instructions into a cascade.
• Build a small internal eval set on your real data. Benchmark rates under 2% can become 10 to 40 times worse in production workflows, so vendor scores won't predict your reality.
• Route destructive actions — deletions, payments, sends — through human approval regardless of agent confidence.

This is why we're building SkillzBench the way we are: every skill version-pinned, security-scanned, and benchmarked in a sandbox with and without the skill, so you can see the Lift before an agent ever touches your data. When one hallucination can cascade through three agents into deleted financial records, "it seemed to work" isn't a deployment standard.

Your Hallucination-Reduction Playbook

None of this works as a one-off fix — hallucination reduction is a repeatable process. Here's the playbook that emerged from our testing, grounded in what the research actually supports.

Rewrite your prompts with ICE and an explicit escape hatch. Microsoft's engineering guidance recommends structuring prompts as Instructions, Constraints, and Escalation: give a direct ask, add hard constraints like "answer only from the retrieved documentation," and define a fallback such as "If any information is missing, reply with 'Insufficient data.'" Repeat key constraints at the beginning of the prompt, where they have the most impact, and keep temperature in the 0.1–0.4 range for factual work. A prompt with no escalation path forces the model to guess — and models are trained to guess.

Ground every answer in curated sources, then audit them. Retrieval-augmented generation essentially forces a model to source answers from trusted material instead of free-associating from its parameters. But RAG is not a silver bullet: a study of six LLMs on the RGB benchmark found models still struggle to reject false retrieved information and integrate it correctly. Curate your grounding data, filter by metadata, and re-audit regularly for outdated content.

Penalize confident errors in your own testing. OpenAI's SimpleQA data makes the case vividly: a model that abstained 52% of the time cut its error rate to 26%, versus 75% for a guessing model with nearly identical accuracy. If your internal evals reward lucky guesses, your chatbot will learn to guess. Give partial credit for honest "I don't know" answers and route low-confidence outputs to human review queues — automated checks miss subtle errors that humans catch.

For agents, verify every step before deployment. A single chat completion has one hallucination opportunity; a multi-step agent has one at every step, and agentic tasks consume roughly 1,000x more tokens than simple chat. IBM documented the cascade: one agent fabricated a storage-bucket metric, the next agent flagged the bucket for decommissioning, a third deleted it — the only copy of financial records under legal hold. That's why we only run agent skills that are version-pinned, security-scanned, and benchmarked with and without the skill, so we can see the actual Lift before trusting one. This is precisely the gap SkillzBench is building for: a free, independent directory where every skill is pinned to a commit SHA, validated, security-scanned, and tested in a sandbox with the same model — with the waitlist live now.

Skip the traps. Don't pick a model by its benchmark hallucination score — top models scoring under 2% on grounded summarization can hit 16–33%+ on open-domain tests, and enterprise workflows push rates 10 to 40 times higher. Don't tune your chatbot's personality to be maximally agreeable, either: research links friendlier personas to reduced accuracy, since agreeable models are more inclined to agree with the user.

Hallucinations can be reduced, not eliminated. The teams that win are the ones that measure defensibility in their own workflows — not on someone else's leaderboard.