The pilot plan worksheet
One workflow, mapped end to end — data in, transforms, integrations, outputs — with a baseline, a success metric, and a written decision rule. Fill it in with your team in ninety minutes. Save it, print it, or download it. Nothing leaves your browser.
How to use this
- Session 1 (~60 min): sections 1–3. Bring someone who has access to the workflow's data.
- Session 2 (~45–60 min): sections 4–8. Bring the decision-maker who signs the go/no-go.
- Fill in what you know. Mark the rest "unknown" — it's data, not failure. Only one section is a hard block: the baseline.
- When done: print it or download the JSON. Attach it to the pilot's project page. When the decision date arrives, you don't argue about whether it worked — you check what was written.
Setup
~10 minThe five fields that make every other section work. Two of them are the ones most worksheets skip: a named owner and a named sponsor. Without both, nothing after this page matters.
Workflow map
~30 minFour categories, mapped end to end: data in, transforms, integrations, outputs. Getting data in right — both at start and mid-stage — is the gate. If you don't know what enters, you can't instrument what leaves.
| Source | Owner | Access notes |
|---|---|---|
Baseline — the pre-pilot state
~20 minThree metrics that describe how the workflow performs *today*. Without them, you cannot prove the pilot worked — you can only assert it. This is the one section that is a hard block: if the baseline is mostly unknown, do not start the pilot.
| Metric | Current value | How measured |
|---|---|---|
Success metric and decision rule
~15 minWrite down what "worked" means, and what you will do with each outcome. Written now — before the pilot runs — this is the difference between a decision and a debate. Written after, it's just a rationalization.
Scope boundary
~10 minName what's in scope, and — more importantly — what's out. The out-of-scope list is the less popular half and the more important one. It is the discipline that keeps a pilot from silently growing into three pilots and failing all of them.
Human-in-the-loop posture
~10 minFully automated and human-in-the-loop are not attitudes. They're a decision about which errors are recoverable and which are not — and about how much review burden the workflow can bear. Name the posture, and if there's review, name the threshold.
Risk screen
~10 minThree risks, ranked by (likelihood × impact). Not a full risk register — just the three things that would actually sink this pilot if they happened. Name the mitigation, not the concern.
| Risk | Likelihood | Impact | Mitigation |
|---|---|---|---|
Sign-off
~5 minTwo names and two dates. If the owner and the sponsor can't both sign, the pilot isn't ready — the worksheet has done its job by surfacing that before you spend the eight weeks.
See a filled-in example (insurance · claim summaries)
This is the case study from the hub's regional insurer case, written up as a completed pilot plan. It shows what "done" looks like — not because it's aspirational, but because it's how a real one reads.
Setup
- Workflow
- Drafting claim summaries for adjusters at a regional insurer.
- Description
- Time to first decision drops from 8 minutes to under 5 without reducing decision quality or rework rate.
- Owner
- Priya Raman · VP Claims Operations
- Sponsor
- Marcus Solis · COO
- Duration
- 8 weeks · decision date set
Workflow map
- Data in (start)
- Claim file (PDF), adjuster's intake notes (free text), policy document (PDF), historical decisions for this claim type (from the customer DB).
- Data in (mid-stage)
- Adjuster's category call after the first summary draft; the review checklist status.
- Transforms
- Extract claim facts → summarize the damages narrative → cross-reference the policy → draft the summary → flag for adjuster review.
- Integrations
- Claims management system (read); adjuster UI (write draft); audit log (write every draft + edit).
- Outputs
- Drafted summary (text) delivered to the adjuster; decision-support packet; audit record.
Baseline
- Time to first decision
- 8 min · median, last 90 days, claims system
- Rework rate
- 22% · reopened or escalated within 7 days
- Adjuster adoption
- 71% · weekly active use, top 2 regions
Success metric and decision rule
- Primary
- Time to first decision · target under 5 min
- If hit target
- Scale to the remaining regions within 4 weeks; a second claim-type pilot funded from the same envelope.
- If miss
- Stop within 2 weeks. Do not modify the target. One-page retro of what the workflow map missed goes to the claims ops archive.
- If close (5–6 min)
- One 3-week retune with the adjuster review checklist, then re-measure. If still 5+, apply the stop rule.
Scope boundary
- In scope
- Auto comprehensive claims only · US East region · 2 adjuster teams · human review on every draft.
- Out of scope
- Liability decisions · payment authorization · non-auto claims · any CRM data-model change.
- Scope change rule
- Owner + sponsor in writing, once max, before the pilot's midpoint.
HITL posture
- Posture
- Human-in-the-loop on every action (first 4 weeks), then automated with exceptions if rework holds under 15%.
- Exception trigger
- Any citation outside the current policy version · any reference to a prior claim · any gap in the claim file.
- Reviewer
- Senior adjuster · escalation to team lead within 4 hours.
Risk screen
- Hallucinated policy citation
- Likelihood: high · impact: high · Mitigation: citation check on every draft, HITL review before final.
- Adjuster over-reliance
- Likelihood: medium · impact: medium · Mitigation: weekly calibration on 20 sampled drafts; edit-rate tracked per adjuster.
- PII handling across systems
- Likelihood: low · impact: high · Mitigation: PII redaction at ingestion; data-retention policy signed by risk before start.
Sign-off
- Owner
- Priya Raman · signed
- Sponsor
- Marcus Solis · signed, subject to legal review of PII flow (completed before start)