Test Your Own Workflow

A workflow you have not tested is a guess. Test it on a few real jobs, score what comes back against checks you wrote down first, fix one thing, and run it again. Six steps, shown on one template's run, with two sheets to keep your own.

Sample results: runs and scores on this page are illustrations while testing is set up. How we test

  • Test-Set SheetChoose your test cases and write what good looks like, before you run.Download (test set)
  • Run-Record SheetRecord one run: what each check found, the score, the fix and the re-run.Download (run record)

Six Steps, One Worked Run

Write a summary of a long document

Step 1 of 6: Build a Test Set

Pick three to five real jobs for your workflow: a typical one, an easy one and one with a complication. Keep the source material beside each, and write down what a good result looks like before you run anything.

The Worked Run

Sample

The template is Write a summary of a long document. Its test set has three cases:

  • Typical: A program's year-two report for a finance committee
    A structured report of about 600 words with six headed sections, many figures and two named people; long enough that the summary must choose what to keep.
  • Easy: A short office memo about a new recycling routine
    A one-page memo of about 130 words with clear headings and only a few figures; the summary is short and nothing in it is easy to get wrong.
  • Tricky: A dense study summary with no headings, mixed number styles and a revisionUsed in this run
    About 450 words of running prose with no headings, numbers as digits and words, a figure revised partway through, ranges, and two similar names; tests the map step, the number check and the name check.

Step 1 of 6: Build a Test Set

Six Rules For A Fair Test

  1. 1
    Change one thing at a time
    Same input, same checks, same version. If two things changed, you cannot say which one helped.
  2. 2
    Keep the failures
    A test set of only easy jobs proves little. Keep the case that failed, and run it every time.
  3. 3
    Decide what good is first
    Write the criteria before the run. Criteria written after you have seen the output tend to fit it.
  4. 4
    Count the same rows every time
    Do not drop or soften a criterion after a failure. If it was wrong, change it, say so in the record, and re-score the earlier runs.
  5. 5
    Leave judgment to a person
    A check about fairness, tone or what is missing needs a reader. Record it as your read, not as met.
  6. 6
    Re-run everything after a fix
    A fix for one case can break another. Run the whole test set again before you call it done.

Templates with these checks built in:Write a summary of a long documentTurn meeting notes into decisions and actionsScore work against a rubric

More in Quality

Quality home
  • Spot the mistake
    Nine AI summaries with planted mistakes, three from nonprofit and consulting work. Find them, then see why.
  • Prompt before and after
    Slide from a vague prompt to a well-built one and see what came back.
  • Watch a check run
    See a template check a draft, one check at a time, and the four kinds of check.
  • Trace the number
    Click a claim and find the source line it came from, or does not.
  • Diagnose and fix
    Pick what went wrong in a draft, then see where it is and what good looks like.
  • Check it yourself
    Paste your own AI output and run a template's checks on it, in your browser.
  • Common AI mistakes
    Six common AI mistakes, each with an example and how to catch it.
  • Writing a good check
    Turn "make sure it is right" into a check anyone can run.
  • Scoring guides
    Rate work against what good looks like. Three free guides to download.