How to fact check AI answers in five steps
This method works with any chatbot. It takes a few minutes for a normal answer and is worth it for anything with consequences.
Pull out the checkable claims
Names, numbers, dates, quotes and citations. These are where hallucinations hide and where errors cost the most.
Ask a different model
Give a model from another vendor the question and the answer, and ask it to flag errors, outdated facts and missing context. Or compare AI models side by side.
Go to the primary source
A court database, the official statistic, the paper itself. A second AI narrows what to check; it doesn't replace the source.
Confirm citations exist
Open every reference. Check that it exists and that it actually says what the answer claims it says.
Stop on disagreement
If two models disagree on a fact, verify it before using it. For legal, medical or financial work, add a qualified human reviewer.
Auto Verification runs step 2 for you after every answer and shows a badge. You still do steps 3–5 for anything that matters.
How it worksReal cases: what unchecked AI answers have cost
Widely shared figures such as "$67.4 billion in global losses" come from marketing compilations without published methods, so we leave them out. These cases are on the public record.
In Mata v. Avianca, the lawyers' March 2023 filing cited six court decisions that did not exist. Asked whether the cases were real, ChatGPT said they were and could be found in "reputable legal databases such as LexisNexis and Westlaw." In June 2023 Judge P. Kevin Castel called the legal analysis in one fake opinion "gibberish" and fined the lawyers and their firm $5,000.
The pattern has repeated since, from courtrooms to consulting reports: a confident AI answer, no independent check, and a public correction. The fix is not to stop using AI. It is to stop trusting a single unchecked answer.
How often does AI get facts wrong?
It depends heavily on the task and the model. Two well-documented measurements:
| Test | Hallucination rate | What it shows |
|---|---|---|
| Specific legal questions GPT-3.5, PaLM 2, Llama 2 · Stanford, 2024 | 69–88% at least 75% on a court's core holding | Legal language is easy to imitate, so invented citations look real. |
| Questions about people (PersonQA) OpenAI o3 and o4-mini system card, April 2025 | o1: 16% · o3: 33% · o4-mini: 48% | Newer reasoning models can hallucinate more on simple facts, not less. |
Read these numbers carefully. Both tests use hard question sets and models that have since been replaced, and newer models do better on many benchmarks. But no published rate tells you whether the answer in front of you is right. That is why you check the answer, not the model's reputation.
Why models hallucinate, in short
A language model is not a database lookup. It predicts the most plausible next words. On common topics, plausible and true usually overlap. Hallucinations appear in the gap: rare topics, events after the training cutoff, and things that have a familiar shape, like a case citation, whether or not the case exists.
That is also why "are you sure?" doesn't work. The most plausible reply to that question is "yes." The confidence is generated, not measured.
Why a second model from another family helps
Cross-family verification works best
A December 2025 study of LLMs as verifiers found that verification across model families is more effective than either self-verification or verification within the same family. Gains were largest on math and logic tasks.
Benchmarks use mixed judges for a reason
Google DeepMind's FACTS Grounding benchmark scores answers with three judges (Gemini 1.5 Pro, GPT-4o and Claude 3.5 Sonnet) to reduce the bias of a judge favoring answers from its own model family.
Different training, different blind spots
Models from one vendor share training data and habits, so they tend to fail in the same places. A model from another vendor often catches what the first one missed, though not always.
How MultipleChat Auto Verification works
After a response is delivered, MultipleChat can automatically run it through a factual accuracy check using Google Gemini. The check looks for factual errors, outdated information and missing context, then shows a badge on the response.
Your model answers
You chat with ChatGPT, Claude, Grok, Perplexity or Gemini as usual. The answer is delivered first, so you aren't kept waiting for the check.
Gemini checks it
Gemini receives your original question and the answer. In a single call it decides whether verification is appropriate for this type of content and, if so, what is correct and what issues exist.
You see a badge
Verified if no issues are found, or Issues found with a brief description of the problem.
Smart skip
Verification is skipped automatically, saving credits, where a factual check adds little:
Turning it on
Auto Verification is off by default. Turn on Auto-verify answers in the tools menu next to the message box (paid plans). Once it's on, every answer is checked right after it's delivered, and each check counts as one message.
Source: MultipleChat documentation. When you chat with Gemini itself, the check comes from the same model family. For a cross-family view, use Compare Mode.
Worked example: a fabricated citation
An illustrative example of the failure that sank Mata v. Avianca. The case names below are invented for this page.
"Cite three U.S. federal cases on equitable tolling in personal injury claims involving international flights."
"Three relevant precedents are Henderson v. KLM Royal Dutch Airlines, 412 F.3d 332 (2d Cir. 2005); Cordova v. Avianca Airlines, 549 F. Supp. 2d 1208 (S.D.N.Y. 2008); and Patel v. Air India, 627 F.3d 415 (9th Cir. 2010)..."
Formatted like real citations, stated with full confidence.
"The three cited cases could not be confirmed and appear to be fabricated. Verify any citation in a legal database before relying on it."
The flag tells you where to look. Confirming the law is still your job, or your lawyer's.
Where verification still fails
Shared blind spots
If both models learned the same wrong fact, the checker confirms it. Cross-family checking reduces this risk; it doesn't remove it.
Brand-new information
Neither model can verify an event it has never seen. For current facts, use web search or Research mode with sources.
Errors of degree
A checker catches invented facts more reliably than a figure that is slightly off, such as 30% where the truth is 26%.
When to turn Auto Verification on
| Use case | Recommendation | Why |
|---|---|---|
| Legal research | ✅ On, plus a legal database | Citation fabrication is the best-documented failure. |
| Medical and health questions | ✅ On, plus a clinician or official guidance | An error can cause harm, not just embarrassment. |
| Financial figures | ✅ On, plus the filing or report | Numbers are easy to state confidently and wrongly. |
| Citations, statistics, quotes | ✅ On | Each one is a checkable claim. |
| People, dates, history | ✅ On | Even recent reasoning models miss person-specific facts often. |
| Code | ⏭️ Skipped automatically | Run and test the code instead. |
| Creative writing, opinions, brainstorming | ⏭️ Skipped automatically | Truth is not the relevant measure. |
| Math, how-to, definitions, grammar | ⏭️ Skipped automatically | A factual check adds little here. |
Get a second opinion on every answer
Chat with ChatGPT, Claude, Grok or Perplexity and let Gemini check each answer for errors, outdated facts and missing context. Available on paid plans.
Frequently asked questions
What is an AI fact checker?
An AI fact checker is a second AI model that reviews an answer for factual errors, outdated information and missing context before you rely on it. It works best when the checker comes from a different model family than the model that wrote the answer.
How do I fact check AI answers?
List the checkable claims (names, numbers, dates, quotes and citations), have a model from a different family review them or compare several models side by side, then confirm anything important in a primary source. Where models disagree, verify before you act.
Can an AI fact-check its own answer?
Only weakly. A 2025 study of LLM verifiers found that verification across model families is more effective than self-verification or verification within the same family. A model asked whether it is sure tends to repeat its own answer.
How does MultipleChat Auto Verification work?
After a response is delivered, Gemini receives your original question and the answer. In a single call it decides whether verification is appropriate for that type of content and, if so, what is correct and what issues exist. The response then shows a Verified badge, or an Issues found badge with a brief description.
Is Auto Verification on by default?
No. Auto Verification is off by default. Turn on Auto-verify answers in the tools menu next to the message box; it is available on paid plans. Once it is on, every answer is checked right after it is delivered, and each check counts as one message.
Does Auto Verification check every answer?
No. It skips content where a factual check adds little, which saves credits: simple math and calculations, code generation and debugging, creative writing, opinions and subjective questions, how-to instructions, and definitions or grammar help.
What if the AI fact checker is wrong?
It can be. When two models share the same blind spot, both can be wrong in the same way, so a badge is a signal, not proof. For anything you will publish, file or act on, confirm key claims in a primary source and keep a human reviewer involved.
Have AI hallucinations caused real damage?
Yes. In Mata v. Avianca (2023), a federal judge fined two lawyers and their firm $5,000 after a brief cited six non-existent court decisions generated by ChatGPT. In 2025, Deloitte Australia agreed to partially refund the A$440,000 fee for a government report that contained a fabricated court quote and references to research papers that do not exist.
Sources
- Mata v. Avianca, Inc., No. 1:22-cv-01461 (S.D.N.Y. June 22, 2023), sanctions opinion; reported by the ABA Journal.
- Smith v. Farwell (Mass. Super. Ct., Feb. 2024); reported by the Maryland State Bar Association.
- Associated Press, "Deloitte to partially refund Australian government for report with apparent AI-generated errors," October 7, 2025 (via CTV News).
- Dahl, Magesh, Suzgun and Ho, "Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models," Stanford RegLab and HAI, 2024 (summary).
- OpenAI, o3 and o4-mini system card, April 2025, PersonQA evaluation (reported by Techzine).
- Lu, Teehan, Jin and Ren, "When Does Verification Pay Off? A Closer Look at LLMs as Solution Verifiers," arXiv:2512.02304, December 2025.
- Google DeepMind, "FACTS Grounding: A new benchmark for evaluating the factuality of large language models," December 17, 2024.
- MultipleChat documentation: Auto-Verification.