Common AI mistakes

AI writes smoothly, so its mistakes look like facts. These six turn up again and again. Learn to see them and you will catch most of what matters.

  1. 1Wrong or made-up numberA figure that is not in the material, or one that was changed on the way.
  2. 2Cause and effect claimedTwo things that happen together are described as one causing the other.
  3. 3Made-up sourceA study, expert, standard or quote that was never in the material.
  4. 4Merged figuresNumbers from different questions, groups or years are blended into one.
  5. 5Missing partSomething the reader needs was left out, or an open gap was filled with a guess.
  6. 6Scope creepThe AI adds something that was not asked for or not in the material.

What each looks like, and how to catch it

Practice: spot the mistake

  • Wrong or made-up number

    A figure that is not in the material, or one that was changed on the way.
    The AI wrote: The ad budget was approved at $20,000.
    The source: The notes say $2,000. A zero was added.
    How to catch it: Put each number next to the line it came from. If you cannot find the line, the number is a problem.
    Kind of check: Compare to the source
    Find one in a real summary

  • Cause and effect claimed

    Two things that happen together are described as one causing the other.
    The AI wrote: The new meeting schedule caused the rise in workload complaints.
    The source: The survey shows the two went up together. Nothing in it says one caused the other.
    How to catch it: Look for "because", "caused", "led to" and "so". Ask what in the source shows the link, not just the timing.
    Kind of check: Checklist
    Find one in a real summary

  • Made-up source

    A study, expert, standard or quote that was never in the material.
    The AI wrote: This matches the 2023 Workplace Benchmark Study.
    The source: No benchmark study appears anywhere in what the AI was given.
    How to catch it: Every named source, study or quote must be findable in what you gave the AI. If it is not, remove it or find the real one.
    Kind of check: Compare to the source
    Find one in a real summary

  • Merged figures

    Numbers from different questions, groups or years are blended into one.
    The AI wrote: 72% of staff feel supported by their manager.
    The source: One question says 54%. A different question says 90%. The 72% is not in the survey.
    How to catch it: When a number looks tidy and you cannot find it, check whether it is an average or a blend of two lines.
    Kind of check: Compare to the source
    Find one in a real summary

  • Missing part

    Something the reader needs was left out, or an open gap was filled with a guess.
    The AI wrote: Priya will book the venue.
    The source: The notes say no one is sure who books the venue. The AI should have flagged the gap.
    How to catch it: List the parts the document must have, then tick each one off. Gaps should be flagged, never filled.
    Kind of check: Checklist
    Find one in a real summary

  • Scope creep

    The AI adds something that was not asked for or not in the material.
    The AI wrote: The board asked for weekly updates on the launch.
    The source: The notes never mention the board.
    How to catch it: Ask of every sentence whether it can be traced to the material. Anything extra is a person-look item at best.
    Kind of check: A person looks
    Find one in a real summary

An order for checking

  1. 1
    Numbers and names against the source
    They are the easiest to check and the costliest to get wrong. Check them first.
  2. 2
    Claims and causes
    Look for words like because, caused and led to. Ask what in the source shows the link.
  3. 3
    What is missing or added
    List the parts the reader needs, tick them off, then look for anything extra.

More in Quality

Quality home
  • Spot the mistake
    Six AI summaries with planted mistakes. Find them, then see why.
  • Prompt before and after
    Slide from a vague prompt to a well-built one and see what came back.
  • Watch a check run
    See a template check a draft, one check at a time, and the four kinds of check.
  • Trace the number
    Click a claim and find the source line it came from, or does not.
  • Diagnose and fix
    Pick what went wrong in a draft, then see where it is and what good looks like.
  • Check it yourself
    Paste your own AI output and run a template's checks on it, in your browser.
  • Writing a good check
    Turn "make sure it is right" into a check anyone can run.
  • Scoring guides
    Rate work against what good looks like. Three free guides to download.