AI pilot readiness checklist

01 Make one data source usable first

Pick the data your use case needs, give it an owner and bring it into one governed place. An AI project on scattered data spends most of its budget on cleaning.

02 Choose one process and measure it

Name the single process with the best value for effort and record its volume, time and error rate today. Without a baseline you cannot say later whether the pilot paid off.

03 Name a business owner and a technical owner

A pilot without an owner stops after the demo. Set aside time for both roles, or bring in senior data and AI leadership for a few days a week to start.

04 Write down the rules before the first model

List which data may leave your tenant, who approves AI tools and who answers privacy questions. A one-page rule is enough to start and prevents a stop later.

05 Scope a pilot with one budget and one measure

Agree a small budget, a decision date and one measure that decides success or stop. A fixed-scope pilot on one document stream is a realistic first shape.

06 Plan for AI that runs where your data may stay

If your data cannot go to an external AI service, use open-weight models hosted in a European region, on your own servers or on Azure in an EU region, so the use case is not blocked.

07 Add evaluation and shared building blocks before project two

With the basics in place, the risk shifts to quality and repeatability. An evaluation set with a regression gate, guardrails and tracing let several use cases share one frame.

08 Add an evaluation set with a regression gate

Collect real examples and expected answers, score every change against them and block a release that scores worse. This is the single biggest reliability gain.

09 Put guardrails between the model and the world

Validate input and output, limit each agent's tool permissions to the minimum and require human approval for risky actions. Keep a fallback route if the model fails.

10 Trace every request

Record prompts, tool calls, responses, cost and latency per request so you can open one trace and replay it. Alert on quality and cost, not only on errors.

11 Move shared building blocks into one versioned harness

Put prompts, tools, evaluators and model configuration in a shared, versioned library so new use cases start on the same frame and the model can be swapped by configuration.

12 Release AI changes through a pipeline

Version prompts, model and tool definitions together, deploy through a reviewed pipeline with an evaluation gate, and be able to roll back quickly.

13 Get an outside review of failure modes and cost

Strong harnesses still have blind spots. A short external review, or senior AI leadership for a few days a week, tests the weak points before users do.