• Checklist
  • 3 min read

Seven checks before an AI pilot goes live

A review sheet for the meeting where a pilot asks to become a production system. Each check has a test and a pass mark, and the evidence behind it.

Olivia18 September 2026 · 3 min read

This sheet goes with our report on where AI pilots stall. It is meant to be printed and used in one meeting, with the sponsor, the business owner, the engineer and someone from risk in the room. Record a pass or a fail against each check, and a retest date for every fail.

Check 1

Someone in the business owns the workflow

McKinsey found that nearly three-quarters of its high performers had redesigned workflows because of AI, against about a quarter of other respondents.1 Redesigning a workflow takes someone with authority over it.

Test
Name the person who signs off changes to the process the system touches.
Pass
That person exists, has agreed in writing and has time set aside for a weekly review.

Check 2

The data holds up beyond the sample

Gartner expects organisations to abandon 60% of AI projects through 2026 where the data is not AI-ready, and found 63% of organisations unsure they have the right practices, or sure they do not.2 In Informatica’s 2026 survey, 57% of data leaders called data reliability a key barrier to production.3

Test
Run the system on a fresh sample that nobody prepared by hand.
Pass
Accuracy on the fresh sample stays inside the range the business agreed, and every preparation step is automated.

Check 3

Each business term has one definition

Gartner argues that agents cannot work accurately without a clear understanding of the relationships and rules in an organisation’s data.4 A system answering questions about revenue or customers needs one definition of each.

Test
List the business terms the system uses and find where each one is defined.
Pass
Every term has one definition, one owner and one implementation.

Check 4

It is affordable at ten times pilot volume

One respondent in five in McKinsey’s 2026 survey said their organisation was limiting AI use because of operating costs.1 Gartner lists escalating costs among its reasons for expecting many agentic projects to be cancelled.5

Test
Multiply pilot usage by ten and price it, including review time and data preparation.
Pass
The cost per run at that volume fits the business case with room to spare.

Check 5

It has a risk classification on file

Under the EU AI Act, high-risk obligations for Annex III systems apply from 2 December 2027, and employment uses such as recruitment and performance evaluation are on the list.6 Article 50 transparency duties have applied since 2 August 2026.7

Test
Classify the system against the AI Act and record the reasoning.
Pass
A named person has signed the classification, and Article 50 disclosure and marking are in place where they apply.

Check 6

Wrong answers have somewhere to go

Pilot users put up with errors and mention them in passing. Production users meet the same errors with nobody to tell. Salesforce found that 89% of data and analytics leaders with AI in production had seen inaccurate or misleading outputs.8

Test
Ask where a user reports a wrong output and who sees the report.
Pass
A reporting route exists inside the tool, and a named reviewer responds within an agreed time.

Check 7

It learns from corrections, and someone can switch it off

MIT NANDA names learning capability as the main barrier it found: tools that do not keep feedback or improve with use.9 A system that never improves repeats its mistakes at larger volume.

Test
Make the same correction five times over two weeks and see whether it sticks.
Pass
Corrections persist, someone reviews what the system has learnt, and a named person can switch it off within minutes.

Sources

All sources were published in 2025 or 2026. The MIT NANDA report is cited through trade press coverage, which is named in the entry.

  1. The state of AI in 2026: On the road to ROIMcKinsey & Company, 25 August 2026
  2. Lack of AI-Ready Data Puts AI Projects at RiskGartner, 26 February 2025
  3. New Global CDO Report Reveals Data Governance and AI Literacy as Key Accelerators in AI Adoption (CDO Insights 2026)Informatica, 27 January 2026
  4. Gartner Says Lack of Semantics Causes Inaccurate Artificial Intelligence Agents and Wasted SpendingGartner, 11 May 2026
  5. Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027Gartner, 25 June 2025
  6. The Digital AI Omnibus: deferral of high-risk AI obligations under the AI ActDLA Piper, 30 June 2026, updated 10 August 2026
  7. The Digital Omnibus and the postponement of high-risk obligations to December 2027Praxikon, 2026
  8. Study: 84% of Technical Leaders Need Data Overhaul for AI Strategies to Succeed (State of Data and Analytics)Salesforce, 4 November 2025
  9. MIT Report Finds Most AI Business Investments Fail, Reveals "GenAI Divide" (coverage of MIT NANDA, The GenAI Divide: State of AI in Business 2025)Virtualization Review, 19 August 2025

Keep reading

More from Resources

Report

Where AI pilots stall on the way to production

A stage-by-stage reading of MIT, Gartner, IBM, BCG and McKinsey research on why pilots stop short of production.

Olivia/3 Sep 2026/6 min
Explainer

Agentic AI in the enterprise: a working explainer

A plain account of AI agents: four tests for the word, how the loop works and where first projects go wrong.

Shyam/15 Sep 2026/4 min
Calendar

The EU rules calendar for data and AI teams, 2025 to 2028

Every dated EU obligation for data and AI teams from 2025 to 2028, and the first task each one creates.

Shyam/2 Oct 2026/5 min

BIAAS 2027, Amsterdam

Hear it first-hand from the people doing the work

Two days of keynotes, panels and 1-on-1 meetings with senior data, analytics and AI leaders, on 23-24 March 2027.

View the agenda