This website uses cookies

Read our Privacy policy and Terms of use for more information.

The question that separates two data pipelines is not whether the output looks right, but whether anyone can show where a single figure came from. When you weigh up a data or workflow partner, the first thing you see is clean output: records in, clean records out, numbers that look like the numbers you expected. That tells you the pipeline is plausible. It does not tell you whether it is auditable, and the difference only matters on the day someone asks where a figure came from.

A plausible pipeline produces outputs that look right. An auditable one can show its working: for any output, what went in, what happened to it, what changed since last time, and who decided. Both look identical in a demo. You find out which one you bought later, usually under pressure.

The way to tell them apart before you commit is to ask questions a plausible pipeline cannot answer well. They are, in effect, the auditor's questions asked before you sign rather than after. Four do most of the work.

Can you trace one output back to its sources?

Pick a single figure or record from the demo and ask the partner to walk it back: which source records, which transformations, which rules, in what order. An auditable pipeline can do this for any output, not just a prepared example. If the answer is a diagram of the system rather than the path of that one record, you have learned something.

What changes, and how would I know?

Pipelines change: a mapping is corrected, a rule is tightened, a source system adds a field, a matching rule is retuned. Ask how changes are recorded, whether old outputs can be reproduced as they were, and whether you would be told. "We continuously improve it" is not an answer. It is a description of a pipeline whose past you cannot reconstruct.

What happens to the records that don't fit?

Every real dataset has records that fail a rule: a missing field, a date in the wrong format, a duplicate. A plausible pipeline drops them quietly or forces them into shape, and the output looks complete. An auditable one keeps them, counts them and shows them to someone. Ask where the rejects go and who looks at them.

Who decides, and where is that written down?

Some steps in any pipeline are judgements: how to resolve a conflict between two sources, what counts as a duplicate, which record wins. Ask who made those calls, whether they are written down, and who can change them. If the answer is "the system handles it", the judgement still exists. It has just been made by whoever configured the system, without a name attached.

None of these questions is hostile. A partner with an auditable pipeline tends to answer them quickly, because the questions match how they already work. A partner with a plausible one will answer in general terms, offer reassurance, or promise to come back. How they answer tells you as much as what they say.

This is vet in practice: checking data handling and fit before anything goes in, not after something goes wrong. The vetting bar names a pipeline that cannot show where a number came from as one of the failures worth testing for in a person; the same test applies to the system.

Before you sign

Ask the four questions, and ask for one answer to be shown rather than told: trace a single output you choose, live. If the partner can do that comfortably, most of the rest tends to follow. If they cannot, the clean outputs in the demo are not evidence of anything you can rely on later.

If you are weighing a data or workflow partner and want a second view on the answers: [email protected]

❝

TL;DR: A plausible data pipeline produces outputs that look right; an auditable one can show where each output came from. Before committing to a data or workflow partner, ask four things: can you trace one output to its sources, how are changes recorded and would I be told, where do the records that don't fit go, and who made the judgement calls. Then ask them to trace one output you choose, live.

H121Group

Not legal, security or financial advice. Governed performance.

Disclaimer - Information shared here is for educational purposes only. Individuals and business owners should evaluate their own business strategies and identify any potential risks. The information shared here is not a guarantee of success. Your results may vary.

Governed performance for the AI era · Building responsibly, in public · Nothing on this site is financial, legal or professional advice.