0% complete
Saved locally
Tool · Pilot stage

The pilot plan worksheet

One workflow, mapped end to end — data in, transforms, integrations, outputs — with a baseline, a success metric, and a written decision rule. Fill it in with your team in ninety minutes. Save it, print it, or download it. Nothing leaves your browser.

Time
90–120 min · 2 sessions
In the room
Function lead · AI engineer · finance partner
Requires
One workflow. Not three.
You leave with
One decision-ready pilot plan

How to use this

  1. Session 1 (~60 min): sections 1–3. Bring someone who has access to the workflow's data.
  2. Session 2 (~45–60 min): sections 4–8. Bring the decision-maker who signs the go/no-go.
  3. Fill in what you know. Mark the rest "unknown" — it's data, not failure. Only one section is a hard block: the baseline.
  4. When done: print it or download the JSON. Attach it to the pilot's project page. When the decision date arrives, you don't argue about whether it worked — you check what was written.

Setup

~10 min

The five fields that make every other section work. Two of them are the ones most worksheets skip: a named owner and a named sponsor. Without both, nothing after this page matters.

Name the workflow, not the tool. "Drafting first replies to routine tickets" beats "the AI ticket bot."
The before → after. One sentence. If it takes two, the pilot is too broad.
The one person whose name is on the pilot. Runs it, tunes it, decides with the data.
The one person who funds it and signs the go/no-go. Executive sponsor.
Typical: 6–10 weeks. If your plan needs more than 12, split it into two pilots.
The date the sponsor decides scale / retune / stop. Set it now, in writing, before the pilot begins.

Workflow map

~30 min

Four categories, mapped end to end: data in, transforms, integrations, outputs. Getting data in right — both at start and mid-stage — is the gate. If you don't know what enters, you can't instrument what leaves.

What enters the workflow when it starts? Formats, sources, owners, access. Be specific: "customer email (text), ticketing history (CRM API), and the current knowledge base article (KB)" — not "the data."
What enters partway through? Human inputs (a manager's approval, a SME's note), agent outputs from prior steps, or external signals (a status update, a new email). Mid-stage data is the one teams forget — and the one that breaks reproducibility.
What changes the data? The steps — extraction, classification, drafting, summarization, escalation decisions. Name the decisions the workflow makes, not the model it uses.
Systems, APIs, people, and approvals that sit at the boundaries. What comes in from where, what goes out to where. Also: which systems must be *changed*, not just read from.
What leaves the workflow? Data out, finished products, downstream consumers. And what downstream system, team, or decision depends on the output?
Data sources — name the top three and who owns them
Source Owner Access notes

Baseline — the pre-pilot state

~20 min

Three metrics that describe how the workflow performs *today*. Without them, you cannot prove the pilot worked — you can only assert it. This is the one section that is a hard block: if the baseline is mostly unknown, do not start the pilot.

Name the actual data you'd need to pull. "Better data" isn't an answer; "handle time from the last 500 tickets in the ticketing system" is.
Three baseline metrics · current state
Metric Current value How measured

Success metric and decision rule

~15 min

Write down what "worked" means, and what you will do with each outcome. Written now — before the pilot runs — this is the difference between a decision and a debate. Written after, it's just a rationalization.

One metric. Not a dashboard. The one you'd stake the program on.
The number that means "worked." Not the ceiling — the bar.
Same method as the baseline. If it's different, the comparison is invalid.
The action if the target is met. Be specific about the next step: expand to which teams, fund which follow-on.
The action if the target is missed. "Keep trying" isn't a rule.
Optional but useful: the range where you'd retune and retest, rather than scale or stop. Say how long the retune gets.

Scope boundary

~10 min

Name what's in scope, and — more importantly — what's out. The out-of-scope list is the less popular half and the more important one. It is the discipline that keeps a pilot from silently growing into three pilots and failing all of them.

Both dimensions: what part of the workflow, and which population (team, region, customer segment).
The discipline section. Name what you are choosing not to include, and why — even the tempting items.
The rule for expanding scope mid-pilot. Default: it can't. If you allow it, name the approver and the limit.

Human-in-the-loop posture

~10 min

Fully automated and human-in-the-loop are not attitudes. They're a decision about which errors are recoverable and which are not — and about how much review burden the workflow can bear. Name the posture, and if there's review, name the threshold.

Name the trigger, not the concept. "Uncertainty above 0.7" beats "when the model isn't sure."
Who sees a flagged item next, and in what timeframe.

Risk screen

~10 min

Three risks, ranked by (likelihood × impact). Not a full risk register — just the three things that would actually sink this pilot if they happened. Name the mitigation, not the concern.

Top three risks
Risk Likelihood Impact Mitigation
Name the person and role. "The team" isn't an approver — one person is.

Sign-off

~5 min

Two names and two dates. If the owner and the sponsor can't both sign, the pilot isn't ready — the worksheet has done its job by surfacing that before you spend the eight weeks.

Anything the sponsor agreed to conditionally — budget, resourcing, approval-gating.
See a filled-in example (insurance · claim summaries)

This is the case study from the hub's regional insurer case, written up as a completed pilot plan. It shows what "done" looks like — not because it's aspirational, but because it's how a real one reads.

Setup

Workflow
Drafting claim summaries for adjusters at a regional insurer.
Description
Time to first decision drops from 8 minutes to under 5 without reducing decision quality or rework rate.
Owner
Priya Raman · VP Claims Operations
Sponsor
Marcus Solis · COO
Duration
8 weeks · decision date set

Workflow map

Data in (start)
Claim file (PDF), adjuster's intake notes (free text), policy document (PDF), historical decisions for this claim type (from the customer DB).
Data in (mid-stage)
Adjuster's category call after the first summary draft; the review checklist status.
Transforms
Extract claim facts → summarize the damages narrative → cross-reference the policy → draft the summary → flag for adjuster review.
Integrations
Claims management system (read); adjuster UI (write draft); audit log (write every draft + edit).
Outputs
Drafted summary (text) delivered to the adjuster; decision-support packet; audit record.

Baseline

Time to first decision
8 min · median, last 90 days, claims system
Rework rate
22% · reopened or escalated within 7 days
Adjuster adoption
71% · weekly active use, top 2 regions

Success metric and decision rule

Primary
Time to first decision · target under 5 min
If hit target
Scale to the remaining regions within 4 weeks; a second claim-type pilot funded from the same envelope.
If miss
Stop within 2 weeks. Do not modify the target. One-page retro of what the workflow map missed goes to the claims ops archive.
If close (5–6 min)
One 3-week retune with the adjuster review checklist, then re-measure. If still 5+, apply the stop rule.

Scope boundary

In scope
Auto comprehensive claims only · US East region · 2 adjuster teams · human review on every draft.
Out of scope
Liability decisions · payment authorization · non-auto claims · any CRM data-model change.
Scope change rule
Owner + sponsor in writing, once max, before the pilot's midpoint.

HITL posture

Posture
Human-in-the-loop on every action (first 4 weeks), then automated with exceptions if rework holds under 15%.
Exception trigger
Any citation outside the current policy version · any reference to a prior claim · any gap in the claim file.
Reviewer
Senior adjuster · escalation to team lead within 4 hours.

Risk screen

Hallucinated policy citation
Likelihood: high · impact: high · Mitigation: citation check on every draft, HITL review before final.
Adjuster over-reliance
Likelihood: medium · impact: medium · Mitigation: weekly calibration on 20 sampled drafts; edit-rate tracked per adjuster.
PII handling across systems
Likelihood: low · impact: high · Mitigation: PII redaction at ingestion; data-retention policy signed by risk before start.

Sign-off

Owner
Priya Raman · signed
Sponsor
Marcus Solis · signed, subject to legal review of PII flow (completed before start)

What to do with the finished worksheet

The plan is one artifact in a chain. Here's where it goes next.