Learning objective
By the end of this lesson, you can turn an AI pilot into a bounded PDSA test with a prediction, balancing measure and decision rule.
Why it matters
Without a written prediction, teams reinterpret any result as success. Adoption, output quality and business impact become mixed together, while shifted workload or risk remains unmeasured.
Core concept: a test should challenge an assumption
Before the pilot, write:
- what will improve and by how much;
- for which cases and over what period;
- why the change should cause the improvement;
- what harm or burden might increase;
- what evidence means adopt, adapt or stop.
Run the smallest safe test capable of contradicting the theory.
Visual explanation
Text alternative: a prediction-led plan leads to a bounded test, comparison of expected and observed evidence, and an adopt, adapt or stop decision.
Worked example
A support team predicts that shadow-mode AI classification will correctly route at least 90% of 200 common tickets and identify at least 95% of urgent cases. Reviewer minutes and subgroup error rates are balancing evidence.
If urgent sensitivity misses the threshold, the team does not enable live routing even when average accuracy looks good. It adapts the urgency gate and repeats the test.
Common mistake
Mistake: calling a broad rollout a pilot.
A real test has bounded exposure, a comparison point, explicit safeguards and a near-term decision. “Launch and watch” makes learning slow and rollback costly.
Quick check
Which item belongs in Plan before an AI test starts?
A. A result rewritten after launch.
B. A prediction and adopt-adapt-stop rule.
C. A commitment to scale regardless of evidence.
D. Only the vendor accuracy claim.
Answer: B. The prediction and decision rule protect the integrity of later learning.
Practical prompt
Write one sentence: “For [bounded cases], we predict [change and magnitude] within [time], while [balancing measure] remains within [limit]. We will stop if [condition].”
Summary
- Write a prediction before acting.
- Test the smallest safe informative scope.
- Pair outcome evidence with a balancing measure.
- Compare result with prediction, including deviations.
- Treat adopt, adapt and stop as valid decisions.
Next lesson
Apply the method in Test Support Triage Before Scaling, where average accuracy competes with urgent-case safety.