"Human-in-the-loop" gets used as an answer, not a description. Someone asks whether an AI system is safe to rely on, and the reply is that a human is in the loop — as if the phrase settled the question rather than restating it. It doesn't say what that person can do, what they were shown, or what happens if they say no. It says a person exists somewhere near the process, which is a much lower bar than most people hear it as.
The gap matters because "in the loop" covers two different arrangements, and only one of them does any governing.
The first is approval: a person signs off on a plan, a policy, a workflow, before it runs. That's a real act, but it happens once, at the start, on the plan — not on what the plan produces each time it executes afterwards. We've made a version of this point before: one yes at the top of a chain is consent to a plan, not to what the plan became by the time it acted. The second arrangement is decision: a person looks at a specific output and has the standing, the information and the time to actually judge it before it goes anywhere. Most things called human-in-the-loop are the first, dressed as the second.
Three tests tell you which one you actually have.
Can they say no, and does it stick?
If a rejection gets escalated, overridden, or quietly routed around because the deadline doesn't move, the reviewer isn't a decision point — they're a delay with a job title. Authority that can be appealed away by whoever is impatient that week isn't authority.
Do they have enough to judge it, or only enough to nod?
A person shown a finished, fluent output and asked to approve it in thirty seconds isn't exercising judgement — they're extending trust to whoever built the system, which is a different act wearing the same name. Real review needs the working, not just the conclusion: what the system was given, what it discarded, and what it was uncertain about before the confidence got smoothed out of the final draft.
Do they carry the consequence if it's wrong?
This is the test most organisations skip, because writing it down is uncomfortable. If no specific name is attached to a specific output — if "the system decided" is an acceptable answer to "who decided" — the human in the loop was present without being accountable, which is closer to a witness than a decision-maker.
None of this argues for putting a person in front of every output. Most steps in most workflows don't need eyes on them, and pretending otherwise just trains people to rubber-stamp faster, which is worse than no review at all. The argument is narrower: wherever an organisation is relying on "a human is in the loop" to make a claim about safety or accountability, that claim should survive the three tests above. If it only survives the first — someone, somewhere, once said yes — it isn't a governance control. It's a true sentence that doesn't do anything.
This is what verify means in vet, disclose, verify: not that a human exists in the process, but that a specific person, with real standing and real information, decided — and would be the one asked about it afterwards.
If you're relying on human-in-the-loop as a safeguard and haven't checked which version you actually have, that's worth a short conversation before someone else asks the question for you — [email protected]
Where to put the checkpoint — the workflow-design version of this problem.
What an auditor will ask about your AI system — "who decided, not who approved" as an external evidence question.
We ran our own governance audit on ourselves. It failed. — what happened when we applied this standard to our own process..
TL;DR: "Human-in-the-loop" is often treated as proof an AI system is safe, when it only proves a person exists somewhere near the process. A genuine decision point needs three things a checkbox doesn't have: authority that can't be overridden by impatience, enough information to judge rather than nod, and a named person who carries the consequence if the output is wrong.
Not legal, security or financial advice. Governed AI performance.