AI Pilot Kill Criteria Scorecard
CTA word: SCORECARD
Fill before kickoff. Review weekly. Stopping is a success if the criteria say so.
Pilot identity
Name/ID, owner, start/end, primary metric, baseline value/date.
Hard kill (any one = pause within 48h)
- Safety or compliance incident attributable to the agent
- Unrecoverable bad write to a system of record
- Primary metric moves the wrong way beyond threshold for N days
- Approver queue backlog over limit for N days
- Data freshness breach lasting beyond limit
Soft kill (two or more = executive review)
- Override rate over threshold
- False-action rate (sampled) over threshold
- Time-to-decision not improved vs baseline after week N
- Owner unavailable / no weekly review held twice
- Scope expanded twice without re-baselining
Continue / expand (all required)
- Hard kills clear
- Soft kills <= 1
- Metric improvement vs baseline met
- Audit log complete for sampled week
- Floor / warehouse sponsor still advocates
Weekly score
Track week, metric, overrides %, bad writes, decision, notes for weeks 1-4.
Sign-off
Owner and sponsor signature / date.
Team Nebula / teamnebula.ai