The design question
Not "should a human be involved?" but "where, and on what?" Gates placed by reversibility and blast radius protect you without destroying throughput. Gates placed everywhere produce a system nobody uses.
Human oversight is what makes AI automation acceptable to deploy. Badly designed, it is also what makes it useless. This is how to place it deliberately.
Gate by consequence, not by action type
The common mistake is gating all writes and no reads. The better axis is what happens if it is wrong:
| Action | Reversible? | Blast radius | Gate |
|---|---|---|---|
| Look up a record | — | None | Automatic |
| Draft a reply | Yes | None until sent | Automatic |
| Update an internal note | Yes | Small | Automatic, logged |
| Change a record customers see | Yes, awkwardly | Moderate | Confirm or notify |
| Send an external message | No | Reputational | Always confirm |
| Issue a refund or payment | No | Financial | Always confirm |
| Delete data | No | Potentially severe | Always confirm |
Note that "draft a reply" is automatic and "send it" is gated. That split captures most of the value — the agent does the work of composing, the human does the work of deciding. Gating the drafting step as well would remove the saving while adding no safety.
The three failure modes of badly-placed gates
1. Too many gates — the system is abandoned
If every action needs approval, the human is doing the work with extra steps. Adoption collapses and the project is judged a failure, usually blamed on the AI rather than the design.
2. Too few gates — one incident ends the programme
An agent sends something wrong to a customer, or issues a refund it should not have. The organisational response is rarely "adjust the thresholds" — it is switching the system off entirely.
3. Gates that become rubber stamps
The most insidious. A reviewer approving two hundred near-identical items per day stops reading by item thirty. The gate exists on paper and provides no actual oversight.
Rubber-stamping is a design problem, not a discipline problem. If your review queue is large and repetitive, the fix is raising confidence thresholds so fewer items need review — not asking reviewers to concentrate harder. Meaningful oversight requires the volume to stay meaningful.
Designing the review experience
The interface determines whether oversight is real:
- Show what is uncertain, not everything equally. Highlight the specific field or decision that triggered review.
- Show the evidence — the source document, the retrieved passage, the reasoning.
- Make approval fast and rejection easy — keyboard shortcuts, one-click actions.
- Allow correction, not just rejection. A reviewer fixing one field is far better than sending the whole item back.
- Show the queue size so reviewers can pace themselves.
- Capture why something was rejected — that feedback is how the system improves.
Confidence thresholds as the control
The threshold is what determines review volume. It should be tunable and reviewed with real data:
- Start conservative — more review than you think necessary.
- Measure the override rate. If reviewers approve 98% unchanged, the threshold is too low and you are wasting their time.
- Measure the error rate in the auto-approved band by sampling. If errors appear, the threshold is too high.
- Tune per category. Different document types or request types warrant different thresholds.
- Re-check after changes. A model version change can shift confidence calibration.
A threshold nobody tunes drifts out of alignment with reality. Review the override rate monthly — it tells you whether your humans are adding value or acting as a formality.
Earning autonomy incrementally
The pattern that works organisationally as well as technically:
- Phase 1 — human approves everything. Collect data on what the agent proposes versus what humans decide.
- Phase 2 — auto-approve categories where agreement was near-total, keep review on the rest.
- Phase 3 — widen autonomy on evidence, retaining gates on anything irreversible permanently.
- Ongoing — sample-audit auto-approved items so drift is detected.
Note that step 3 has a permanent floor. Some actions should never be automatic regardless of how reliable the agent becomes, because the cost of the rare failure is not proportionate to the saving.
What to log
- What was proposed, what was approved, and any modification made.
- Who approved it and when.
- Confidence score at the time of the decision.
- Rejection reasons, categorised.
- Time spent in review, so you can see if it is becoming a bottleneck.
That log is also your training signal — the difference between what the agent proposed and what humans chose is exactly the feedback that improves the system.
Designing oversight for an AI deployment? Tell us what actions are involved — gate placement is usually clear once the consequences are mapped. See our agentic AI service, risk management framework, and guardrails to demand.