Human in the loop means a person has the final say at a set point in an automated process. It does not mean a person checks every output. It works when the system holds the cases it is unsure about and sends them to a named approver.
The phrase is common in AI policies and says little on its own. What matters is where the person stands, what they are handed, and whether they can say no.
Three ways to put a person in the loop
| Pattern | What happens | Where it breaks |
|---|---|---|
| Review everything | A person approves every output before it lands | It caps at one person's day. Within a month, approving becomes clicking. |
| Review nothing | The system acts, somebody checks samples later | Errors are discovered by the customer who received one. |
| Review by exception | The system acts when confident and holds when it is not | Only if the threshold is honest and the approver is a named person. |
Where you set the threshold
Every run produces a confidence, whether the system says so or not. Review by exception makes that number explicit and attaches a rule to it: above this line act, below it stop and ask. Where you put the line is the most important decision in the deployment.
Set it too high and everything holds. Held items pile up, and reviewers begin approving in batches without reading. You have paid for automation and got a slower manual process. Set it too low and holds become rare enough that the reviewer stops expecting them, which is when one is missed. The number is not settled by argument. It is found by running a batch of real items, seeing where the system was wrong, and setting the line below that.
The approver has to be a person
The most common failure is not a badly chosen threshold. It is routing holds to a shared inbox, a channel, or a role that four people half own. Work sent to a group waits longer, because no one person owns it. Name an approver for each type of decision, and name their cover. Put both in the record, so it is quick to find who approved something.
What a good hold looks like
The reviewer's time is the expensive resource in this system, so a hold has to arrive finished. A good hold carries the item, the evidence, the reason it stopped and the options. The approver needs one action each way.
- ITEM
- INV-40218, bank detail changed
- WHY HELD
- rule bank-detail-change
- EVIDENCE
- invoice, previous bank detail, the new one
- OPTIONS
- approve or decline
- AFTERWARDS
- sealed, and kept as a labelled example
GatehouseThat is the hold in RUN-7F2K4, the simulated run on Gatehouse, Surehand's control plane. The AP lead declined it at 14:32:19. Read the run on Gatehouse
Five checks that show whether yours is real
- /01
Ask what your hold rate was last week. If nobody knows the number, there is no loop, only a policy.
- /02
Ask to see a run that was refused. Refusals are the proof that a boundary exists at all.
- /03
Ask what happens when the approver is on leave. A control with one point of failure fails on a Friday.
- /04
Ask whether the reviewer's decision goes anywhere. If corrections do not return as labelled examples, you are paying for the same mistake again and again.
- /05
Ask who can lower the threshold, and whether doing so leaves a trace. A control that anyone can loosen without a trace is not a control.
Human in the loop has to be built in from the start. It needs a threshold, a named approver, evidence gathered in advance, and a record of each decision.
Keep reading
How to monitor AI agents in production
Four signals worth watching: quality drift, hold rate, spend per run and latency. Uptime alone does not show whether the answers are still good.
What is AI governance?
Six controls that decide what an AI system may do, on whose authority, at what cost, and with what evidence. A practical guide rather than a policy template.
