How mature is your AI harness?

Your score comes only from your own answers and the rules below. It is not a benchmark against other companies, and we have no data on how others score.

1. Every question adds points to one dimension. Single choice earns the listed points (0 to 4). Each ticked option of a multiple-choice question earns 1 point. A scale answer earns its number minus 1.
2. A dimension score is the points you earned divided by the points available in that dimension, as a percentage.
3. The overall score is all points earned divided by all points available. Every dimension has the same number of questions, so each counts equally.
4. Levels follow the overall score: Starting 0 to 29, Developing 30 to 54, Established 55 to 79, Leading 80 to 100.
5. Recommendations come from the rules listed per check. The weakest dimensions rank first, and the top three are shown.

Points per question

Each answer shows the points it earns on the dimension of the question.

1. 01 / Evaluation How do you know a change did not make your AI feature worse? Up to 4 points

   • We try a few prompts by hand 0 pts
   • A fixed example set, checked by hand 1 pt
   • An automated test set that runs on every change 3 pts
   • An automated test set with scored metrics, thresholds and a gate in the pipeline 4 pts

2. 02 / Evaluation Which kinds of evaluation do you run? Up to 4 points

   • Accuracy against labelled examples 1 pt
   • Rubric or model-based scoring 1 pt
   • Human review of a sample from production 1 pt
   • Adversarial or red-team cases 1 pt
   • None of these 0 pts

3. 03 / Guardrails What stops a bad answer or action from reaching a user or a system? Up to 4 points

   • Nothing in particular 0 pts
   • Instructions in the prompt only 1 pt
   • Input and output checks, such as filters and schema validation 3 pts
   • Checks, plus permission limits on tools and human approval for risky actions 4 pts

4. 04 / Guardrails Which of these do you have? Up to 4 points

   • Prompt injection tests 1 pt
   • Detection and redaction of personal data 1 pt
   • Tool permissions per agent, on least privilege 1 pt
   • A kill switch or fallback route 1 pt
   • None of these 0 pts

5. 05 / Observability What can you see after a request has run? Up to 4 points

   • Nothing 0 pts
   • Application logs 1 pt
   • Traces of prompts, tool calls and responses 3 pts
   • Traces with cost, latency, quality scores and alerts 4 pts

6. 06 / Observability How fast could you find out why one answer was wrong? Up to 4 points

   Scale 1 to 5 earns the answer minus 1 (0 to 4 points).

7. 07 / Reuse How do teams share AI building blocks? Up to 4 points

   • Every project starts from scratch 0 pts
   • By copy and paste 1 pt
   • A shared library of prompts, tools and evaluators 3 pts
   • A versioned shared harness used by several applications 4 pts

8. 08 / Reuse How easy is it to swap the model or the provider? Up to 4 points

   Scale 1 to 5 earns the answer minus 1 (0 to 4 points).

9. 09 / Deployment How do AI changes reach production? Up to 4 points

   • By hand 0 pts
   • By scripts that one person runs 1 pt
   • A pipeline deploys, but evaluation is manual 3 pts
   • A pipeline with an evaluation gate, staged rollout and rollback 4 pts

10. 10 / Deployment Which of these are versioned together? Up to 4 points

    • Prompts 1 pt
    • Model version 1 pt
    • Tool definitions 1 pt
    • The evaluation set 1 pt
    • None of these 0 pts

Recommendation rules

• Evaluation scores below 50 → recommend: Add an evaluation set with a regression gate
• Guardrails scores below 50 → recommend: Put guardrails between the model and the world
• Observability scores below 50 → recommend: Trace every request
• Reuse scores below 50 → recommend: Move shared building blocks into one versioned harness
• Deployment scores below 50 → recommend: Release AI changes through a pipeline
• You answered 3: "Nothing in particular" / "Instructions in the prompt only" → recommend: Put guardrails between the model and the world
• Overall score is 70 or higher → recommend: Get an outside review of failure modes and cost

Levels

• Starting (0 to 29): AI features rely on manual checks. An evaluation set with a regression gate is the highest-value first step.
• Developing (30 to 54): Some practices exist, but changes can still make things worse without anyone noticing. Close the weakest dimension first.
• Established (55 to 79): A solid harness is in place. The gaps are usually in sharing it across teams and in rollout and rollback.
• Leading (80 to 100): Evaluation, guardrails, tracing and release are well covered. An outside review of failure modes and cost is the most useful next step.