Writing · Free lesson
AI English: report variation across repeated runs
A test lead needs a useful observation, not “the AI is random.” Name what stayed the same, how many runs you observed and what actually varied.
Professor Scott MendozaThe short answer
“Across three runs with the same prompt and recorded settings, two responses used bullet points and one used a paragraph.” This reports the sample; it does not establish a general failure rate.
Conditions + sample size + observed difference.
In this fictional test log, the same prompt and recorded settings were used three times. Two responses were bullet lists; one was a paragraph. Across three runs limits the claim to those observations. Varied in format names the difference without claiming that the facts were wrong. No format requirement or correctness judgment is supplied, so do not label one format a failure.
Compare the examples
We kept the prompt and recorded settings unchanged.
Two of the three responses used bullet points.
The format varied in these runs; correctness was not assessed.
Two of three describes this sample. It does not mean two of every three future responses will behave the same way. Different wording or format is not automatically an error.
Build the meaning step by step
Scope the sample
Across three runs
State the shared input
with the same prompt and recorded settings
Describe the variation
two responses used bullet points and one used a paragraph
Try it now
Keep the claim within the evidence
Read all answers and explanations
Which statement accurately reports the supplied log?
Two of the three responses used bullet points.Only the first stays within the observed count.
No required format is supplied. What does a paragraph instead of bullets establish?
The response format differed.It establishes a format difference, not factual error or a universal system judgment.
Which phrase preserves the supplied conditions without adding a cause?
with the same prompt and recorded settingsThe log gives unchanged visible conditions, not evidence of a hidden change or upgrade.
Put it into practice
Write the test note
Report three runs: identical prompt and recorded settings, two bullet lists and one paragraph. Correctness was not assessed.
The AI fails a third of the time.
See a model answer
Across three runs with the same prompt and recorded settings, two responses used bullet points and one used a paragraph. Correctness was not assessed.
A careful observation gives the team useful evidence for its next investigation.
A little more clarity
Common questions
A few useful details before you move on.
Does the same prompt guarantee the same output?
Not in every AI system or setup. Report the actual conditions and observed behavior rather than assuming a guarantee.
The prompt was unchanged, but the format varied in these runs.
Can I use two of three instead of a percentage?
Yes. It makes the small sample visible. A percentage would still need the sample size and would not establish a future rate.
two of three observed responses
Take it into your day
Report what varied in the observed runs; do not invent a universal rate.
English for Careers · Coming Soon
Practice English for applied AI teamwork.
English for AI Engineers: Career Simulation is coming soon. Its planned scenarios move an AI demo toward a bounded, evaluated and observable pilot. Sign up on the course page for its launch news.
Get launch updatesAI Engineers · Career Simulation
Your next step · Available on Udemy
Strengthen your English foundations.
The available English Grammar and English Speaking Master Course develops broader grammar, sentence-building and communication foundations. Careful qualifiers and comparisons help explain observations.
Explore the course English Master Course: English Grammar and English Speaking