A proof of value earns its name by being able to fail. Here is what one has to contain — criteria set before the build, a real workflow, a defined failure path — and what it costs you when any of that is missing.
Most AI pilots succeed. That is the problem with them.
A pilot with no criteria agreed in advance cannot fail, because the criteria get chosen afterwards to fit whatever happened. It produces a demonstration, a good feeling and no basis for a decision. A proof of value is a different thing: it is designed so that the answer is allowed to be no.
What has to be agreed before the build
Five things, in writing, before anyone opens an editor.
- The workflow, named specifically. Not “customer service” but “inbound order status enquiries arriving by web chat and email”.
- The baseline. What the workflow costs today in time, volume and error rate, measured over a period long enough to include a bad week.
- The success criteria. The numbers that would justify going further, and the numbers that would not.
- The failure path. What the system does when a source is unavailable, a check fails or permission is denied.
- The decision-maker. One person who can say yes or no on the organisation's behalf when the result arrives.
The fifth is the one most often missing, and it is the reason so many pilots end without a decision rather than with a negative one.
Why it has to run on a real workflow
A proof of value built on curated sample data proves that the technology works on curated sample data. That was rarely in doubt.
The things that actually decide whether an automation survives production are the ones curated data removes: the badly scanned document, the record with a field nobody has filled in since 2019, the customer whose request does not match any intent, the integration that times out at month end. Those are not edge cases. In most enterprise workflows they are a double-digit percentage of volume.
So run it on real material, including the awkward material, and measure what happens when it cannot complete.
A worked example
A returns team wants to automate eligibility decisions. The proof of value covers one category of product, over four weeks, against a baseline of current handling time and current appeal rate.
The criteria agreed at the start: the automation must reach a decision on at least 70 percent of cases, the appeal rate must not rise, and every declined case must carry the policy clause it was declined under.
It comes back at 74 percent, with a flat appeal rate — but a fifth of declines cannot name the clause, because the policy has an exception that lives in a separate document nobody mentioned. That is a good result. It says the approach works, and it says exactly what has to be fixed before production. A pilot without the third criterion would have reported 74 percent and called it a success.
The assumptions worth stating
- The four weeks are representative. If they include a promotion or exclude a peak, say so in the report.
- The baseline came from system records rather than a manager's estimate.
- The volume tested is large enough that the percentage is not noise.
- The people running the pilot were not also marking it.
What the report has to contain
The findings, including the ones that came back flat. A result that only lists what worked is a sales document, and everyone reading it knows that.
It should also carry a recommendation with a direction: scale, fix and retest, or stop. “Promising” is not a recommendation.
Key takeaways
- A proof of value that cannot fail is a demonstration, not evidence
- Agree the workflow, baseline, criteria, failure path and decision-maker before the build
- Run it on real material, including the material that is hard to process
- Publish the results that came back flat, with the reason
- End with scale, fix, or stop — not with “promising”
If you are scoping one and want to pressure-test the criteria before you commit, that is a good use of a first conversation. Request an AI session and bring the workflow.