Most conversations about AI governance happen in the wrong order. A system gets built, it gets used, and only when something goes wrong — or someone with the authority to ask starts asking — does anyone work out whether it can withstand scrutiny. By then the answer is whatever the system happened to keep, which is usually less than people assume.
An auditor, a regulator, a client's due-diligence team or your own board doesn't need to know how the system works. They need four questions answered, and the answers need to already exist rather than be reconstructed under pressure.
What went into this?
Not the model — the inputs. Where the data came from, what it was checked for before it was used, and whether anyone had the right to use it that way. A system that can't say where its inputs came from can't be vetted, and a system that hasn't been vetted can't be trusted to have been used appropriately, however good its output looks.
This is the question most teams answer worst, because vetting happens once at the start and is rarely written down anywhere the answer can be found again later.
Who decided, not who approved?
Approval and decision aren't the same event. A person who signs off on a plan at the start has approved a plan; they haven't decided what the plan became by the time it acted. An auditor asking "who decided" wants a name attached to the actual output, not a name attached to the process that produced it.
If the honest answer is "the system decided and nobody specifically checked this one," that is the finding, however well the rest of the process was documented.
What did a human actually check, and when?
Not whether a human was "in the loop" — everyone claims that. What they were shown, and whether it was possible to catch an error from what they were given. A reviewer shown only a polished final output isn't verifying; they're forming an impression of something already made to look right.
The distinction an auditor draws is between review that could plausibly have caught a mistake and review that couldn't have, whatever it's called internally.
Can you reconstruct why it did what it did?
Not "can you re-run it" — can you show, after the fact, what it was given, what it produced, and what changed hands in between. This is the one most systems fail on quietly. Logs get kept for debugging, not for reconstruction, and the two purposes overlap less than people assume. If the honest answer here is "not really," none of the other three answers can be checked either — they're just claims.
None of this is exotic. It's the same standard applied to any other operational system that affects people or money: know your inputs, know who's accountable, know what was actually checked, keep a record that survives the question being asked. AI doesn't change the standard. It changes how easy it is to skip it, because the output looks finished whether or not the underlying work was done.
The useful exercise isn't building an AI system and then working out how to defend it afterwards. It's writing down what you'd say to each of these four questions before anyone asks, and treating any answer you can't yet give as a gap rather than a formality.
If you're building something that will eventually face this conversation and want a second pair of eyes on whether the answers would hold up, get in touch — [email protected]
The vetting bar — what "vet" means before anything else happens.
The 15-point gate we run before any AI touches a real person — the checklist version of questions one and three.
What "governed AI" actually means — the operating model these four questions sit inside.
❝
TL;DR: An auditor, regulator or client due-diligence team doesn't need to understand an AI system — they need four questions answered with evidence that already exists: what went into it, who decided (not who approved), what a human actually checked, and whether any of it can be reconstructed afterwards. Most systems fail the fourth question quietly, which means the first three can't be checked either.