Stress-Test the Refund Assistant
Build defensible failure chains and define controls before live financial action is enabled.
24 minutes
Scenario
Northstar Retail is piloting an assistant that reads orders, policy and customer messages, then drafts refund amount, reason and destination for agent approval.
- Your role
- Risk and operations facilitator
- Method
- Failure Mode and Effects Analysis
Evidence pack
Payment changes
3% of orders contain a changed or replacement payment method.
Policy
Refund rules change monthly and two versions remain searchable.
Interface
The draft highlights amount but not destination account or source passage.
Workload
Agents review up to 65 cases per hour during peak periods.
Model behaviour
Answers remain fluent when no supporting policy passage is retrieved.
Permissions
The pilot is read-only; production design proposes direct payment initiation after approval.
History
Last quarter had four manual wrong-account near misses and no financial loss.
Appeal
Customers can contact support, but no dedicated refund-decision correction route exists.
Constraints
- Severity ratings must follow the organisation's agreed financial and customer-impact scale.
- Human review cannot be credited as strong detection without a test of reviewer performance.
- Residual ratings may change only after controls are implemented and verified.
Case steps
Work through each prompt using the evidence pack. Answers and rubric weights stay protected in the interactive flow.
Define two workflow functions and one specific failure mode for each.
For each failure mode, write effect, cause and current prevention and detection controls using e1-e8.
Prioritise the two chains using severity, occurrence, detection and explicit judgement. Explain any high-severity override of the combined score.
Define prevention, early detection and safe-recovery actions with owners and verification evidence.