Ask an adviser whether an AI tool will work for your process and you will usually get an answer. It may be careful, it may be hedged, but it will sound like knowledge. What you will rarely hear is the most useful reply available in most of these conversations: we haven't tested this.
The sentence is avoided for understandable reasons. Advisers are paid to know things, and admitting a gap feels like admitting the fee is not earned. The client wants a decision, and "we haven't tested this" sounds like a refusal to give one. The tools change fast enough that nobody has tested everything, so everyone fills the gap with reasoning from experience, which feels close enough to testing that the difference stops being mentioned.
That last part is where the harm is. Reasoning from experience is useful. It tells you what is likely, what usually goes wrong and where to look first. But it is not the same as having run the tool on work like yours and seen what came back. When an adviser presents one as the other, the client cannot tell which parts of the advice are observed and which are inferred, and so treats all of it as observed. The decision ends up resting on a confidence nobody actually has.
What it costs to leave it unsaid
The cost does not show at the time. It shows later, when the tool meets a case the reasoning did not anticipate: an unusual document, an edge in the data, a change in the vendor's model. At that point the client discovers that the advice was a well-informed guess, and the trust lost is not just in that piece of advice but in every other piece that sounded the same. One unflagged inference can cost the credibility of everything around it.
How to say it so it helps
Three things make the difference between a sentence that ends the conversation and one that moves the decision forward.
Say what you do know, and how. For example, an adviser might say: "Tools of this kind usually handle invoices well; nobody has run this one on your contracts." The client learns where the evidence stops.
Say what a test would look like. A small, specific trial on real work, with a pass mark agreed in advance, is usually cheap next to the decision it informs. Naming it turns a gap into a next step.
Say what it costs not to test. Sometimes the honest advice is that a test is not worth it: the step is low risk, the output is easy to check, the decision is reversible. That is a judgement too, and it should be stated as one.
Our standard files this sentence under vet: the discipline is saying it out loud. It is also disclose: saying which parts are tested and which are not, wherever silence would let someone believe more than we know. It is also why we published our own result when we ran our governance audit on ourselves and it failed. That post says plainly that our own injection test set has been written and never run.
A question worth asking your adviser
Next time you get AI advice, ask one thing: which parts of this have you seen work, and which are you reasoning towards? A good adviser will be able to split the answer cleanly, and will not mind being asked.
If you want a second view on which parts of an AI decision rest on evidence: [email protected]
We ran our own governance audit on ourselves. It failed: the standard tested on our own setup, and where it failed.
AI projects fail at evaluation, not at the demo: why seeing it work once is not testing it.
What "governed AI" actually means: vet, disclose, verify.
TL;DR: In AI advisory work, "we haven't tested this" is often the most useful thing an adviser can say, and the most avoided. Presenting reasoning from experience as if it were tested evidence leaves clients unable to tell observed from inferred, and the cost shows later. Said well, the sentence helps: state what you know and how, describe what a test would look like, and say what skipping it costs.
Not legal, security or financial advice. Governed performance.