# Fact-check against sources: prompt chain Check each claim in a draft against the sources you give, quote the passage that backs or contradicts it, and list what to fix, using nothing from outside those sources. **Made for:** Anyone who writes or approves a report, newsletter or briefing and has the source material at hand. ## Before you start Have these ready: - **The draft to check** (required): Paste the text as it stands now. Every number, date and name in it will be checked. - **The sources to check it against** (required): Paste the text of the reports, records or notes the draft should rest on. If you have several, paste them one after another with a title above each. **How to use this chain** 1. Open a new chat in any AI assistant. 2. Copy the Step 1 prompt, replace every [PASTE HERE: ...] slot with your material, and send it. 3. Keep each answer: later steps ask you to paste it in. Run each step in the same chat or a new one; a new chat keeps steps from blurring together. 4. After a step that has a check, run the check before you move on. ## Step 1: List every claim in the draft Pick out each statement a source could prove or disprove, quoted word for word, before any source is read. Copy this prompt: ````text You are a careful fact checker who reads a draft and picks out what a reader could rely on. Your task is to list every claim in the draft that can be checked against a source. The draft: [PASTE HERE: The draft to check] Rules: - A claim is a statement a source could prove or disprove, such as a number, a date, a name, something that happened, a comparison or a cause. Opinions, hopes and plans are not claims. - Quote each claim from the draft word for word, as short as you can while keeping its numbers and names. Do not reword it and do not correct it while quoting. - List the claims in the order they appear. Do not skip a claim because it looks right, and do not add a claim the draft does not make. - Do not check anything yet. Return one markdown table and nothing else: | # | Claim (quote) | ```` **You get back:** A numbered table of the claims, each quoted from the draft. **Check after Step 1: Every claim is quoted word for word from the draft** Check by eye: Every quote in the Claim (quote) column appears word for word in the draft to check. If it does not pass: Run the step once more, and add what was wrong to the end of the prompt. If it still does not pass, fix it by hand and make a note of it. ## Step 2: Find the passage for each claim For each claim, quote the words in the sources that bear on it, or say that none do. Copy this prompt: ````text You are a research assistant who finds the exact words in a source that bear on a claim. Your task is to find, for each claim below, the passage in the sources that says something about it. The claims: [PASTE HERE: the answer from Step 1 (List every claim in the draft)] The sources: [PASTE HERE: The sources to check it against] Rules: - Quote the passage from the sources word for word, as short as you can while keeping its numbers, dates and names. The same passage may serve two claims. - If the sources give a different figure from the claim, quote that passage. If they say nothing about the claim, write exactly None found. - Use only the sources above. Do not use anything you know from outside them, and do not guess. - Do not judge whether the claim is right. Do not join two passages into one quote. Return one markdown table and nothing else: | # | Claim (quote) | Source passage (quote) | ```` **You get back:** A table that puts a quoted passage from the sources beside each claim, or None found. **Check after Step 2: Every passage is quoted word for word from the sources** Check by eye: Every quote in the Source passage (quote) column appears word for word in the sources to check it against. If it does not pass: Run the step once more, and add what was wrong to the end of the prompt. If it still does not pass, fix it by hand and make a note of it. ## Step 3: Give each claim a verdict Compare each claim with its passage and give one of four verdicts, with the reason in one sentence. Copy this prompt: ````text You are a strict reviewer who compares a claim with the passage that was found for it. Your task is to give each claim one verdict. The claims and their passages: [PASTE HERE: the answer from Step 2 (Find the passage for each claim)] Rules: - Use exactly these verdict words: supported, partly supported, contradicted, not in sources. - supported: the passage says the same thing, with every number, date and name the claim uses. A figure worked out from the passage's own numbers, such as a percentage change, is supported when the arithmetic is right. - partly supported: the passage is close, but the claim rounds it, stretches it or leaves part of it out. - contradicted: the passage gives a different number, date or name. - not in sources: the passage says None found, or the sources say the answer is not known. - In "Why", write one sentence that names the figure or word that decided it. Show the arithmetic if you did any. - Use only the table above. Do not give a claim the benefit of the doubt, and do not use any fact from outside it. Return one markdown table and nothing else: | # | Claim (quote) | Verdict | Why | The Verdict column must say exactly supported, partly supported, contradicted or not in sources. ```` **You get back:** A table that gives each claim one of four verdicts and the reason. **Check after Step 3: Each verdict is one of the four allowed words** Check by eye: Every row of the table has one of "supported", "partly supported", "contradicted" or "not in sources" in the Verdict column. If it does not pass: Run the step once more, and add what was wrong to the end of the prompt. If it still does not pass, fix it by hand and make a note of it. ## Step 4: Write the fact-check report Turn the verdicts into a count, a list of what to fix with wording taken from the sources, and a list of what holds up. Copy this prompt: ````text You are an editor who turns a fact check into a short list of fixes the writer can act on. Your task is to write the fact-check report from the verdicts and passages below. The verdicts: [PASTE HERE: the answer from Step 3 (Give each claim a verdict)] The passages: [PASTE HERE: the answer from Step 2 (Find the passage for each claim)] Rules: - Start with one line that begins "Result:", gives the number of claims checked and says how many got each of the four verdicts. - Under "Fix before you send", write one row for each claim that is not supported, in the same order as the verdicts. If every claim is supported, write one row with None found in every cell. - In "What the sources say", give the source's own words or figure, shortened. If the sources are silent, say so. - In "Suggested wording", rewrite the claim using only the facts and figures in "What the sources say". If the sources cannot back any version of the claim, write "Cut it until a source exists." Do not add a number, name or fact that is not in the passages. - Under "Holds up", list each supported claim as a quote, one line each, with nothing else. Do not rewrite supported claims. Return markdown in this form and nothing else: Result: (the counts) ## Fix before you send | # | Claim (quote) | What the sources say | Suggested wording | ## Holds up (one quoted claim per line) ```` **You get back:** A report with a count line, a table of fixes and a list of the claims that hold up. **Check after Step 4: Every fix row is filled in** Check by eye: Every row of the table has something written in "Claim (quote)", "What the sources say" and "Suggested wording" (a blank or a dash does not count). If it does not pass: Run the step once more, and add what was wrong to the end of the prompt. If it still does not pass, fix it by hand and make a note of it. ## Final checks Run these on the finished result. **Final check 1: Every claim in the fix list is in the draft word for word** Check by eye: Every quote in the Claim (quote) column appears word for word in the draft to check. If it does not pass: Run the step once more, and add what was wrong to the end of the prompt. If it still does not pass, fix it by hand and make a note of it. **Final check 2: The counts add up to the verdicts** Open a new chat (so the check is not swayed by the work) and paste this check prompt: ````text You are a careful reviewer. You do not fix or rewrite the work; you only decide whether it passes one check. The check: The counts add up to the verdicts Count the verdicts in the table below. Do the numbers on the Result line of the report match those counts, and does the number of rows under "Fix before you send" equal the number of claims that are not supported? Answer FAIL and say which count is wrong. The verdicts: [PASTE HERE: the answer from Step 3 (Give each claim a verdict)] The work to check (the answer from Step 4 (Write the fact-check report)): [PASTE HERE: the answer from Step 4 (Write the fact-check report)] Reply with exactly PASS or FAIL on the first line. On the second line, say in one sentence why. If it fails, name the exact item or line that fails. Do not add anything else. ```` If it does not pass: Run the step once more, and add what was wrong to the end of the prompt. If it still does not pass, fix it by hand and make a note of it. **Final check 3: Each suggested wording says only what the sources say** Open a new chat (so the check is not swayed by the work) and paste this check prompt: ````text You are a careful reviewer. You do not fix or rewrite the work; you only decide whether it passes one check. The check: Each suggested wording says only what the sources say Compare each suggested wording in the report with the sources below. Does every number, date and name in it appear in the sources, and does it say no more than the sources do? Answer FAIL and quote any wording that adds a figure or a fact the sources do not give. The sources: [PASTE HERE: The sources to check it against] The work to check (the answer from Step 4 (Write the fact-check report)): [PASTE HERE: the answer from Step 4 (Write the fact-check report)] Reply with exactly PASS or FAIL on the first line. On the second line, say in one sentence why. If it fails, name the exact item or line that fails. Do not add anything else. ```` If it does not pass: Run the step once more, and add what was wrong to the end of the prompt. If it still does not pass, fix it by hand and make a note of it. **Final check 4: You decide what to do with each fix** Ask yourself: Read each row under "Fix before you send". For a claim marked not in sources, decide whether to find a source, soften the claim or cut it, and check the suggested wording against the source yourself before you use it. If it does not pass: Carry on, but write down the problem so whoever uses the result knows about it. ## What you end up with - A count of the claims checked and how each one fared. - A list of the claims to fix, with the passage from the sources and suggested wording for each. - A list of the claims that hold up. ## Changing this chain - This template checks the draft against your sources, not against the world. A claim marked not in sources may still be true; it only means your sources do not show it. - If you have several documents, give each a title on its own line above its text, so you can find a passage again. - To allow rounding, change the line about partly supported in the verdicts step, for example by saying that a number rounded to the nearest hundred counts as supported. - To add a verdict word, add it to the verdicts step's rules and to its check, both. - For a long draft, run the template on one section at a time so every claim gets a passage. --- From Logic Lab (logiclabhq.com). Free to use and adapt, with credit.