Internal audit sends a short request. Confirm that the revenue figure reported in Q1 was computed on the same basis as the figure reported in Q4, and provide support.
There are three ways an organization can answer that, and they sit on a ladder.
Three answers, in ascending order
Assertion. "Our change management process ensures definitions remain consistent across reporting periods." This is a description of intent. It satisfies a light-touch review and it collapses the moment someone asks for the underlying record, because there is no underlying record.
Reconstruction. Two engineers spend a week tracing the metric back through the transformation chain, comparing what they find against a Q1 report, and writing a memo. The memo is usually accurate. It's also expensive, produced by the same team whose work is under review, and unrepeatable. Ask the same question next quarter about a different metric and the week starts again.
Evidence. A query against a change history that returns what the definition said in Q1, what it said in Q4, who changed it if it changed, when, and which reports consumed it at each point. It takes minutes, and someone other than the author can reproduce it.
Most organizations evaluating metadata platforms think they're choosing between vendors. They're choosing which of those three answers they'll be able to give, and the choice is made years before the request arrives.
Why reconstruction keeps getting accepted
Reconstruction survives as a practice because it usually produces the right answer, and auditors accept right answers.
It has three properties worth naming before it gets relied on again.
It's self-attested. The team that built the pipeline is the team reporting on what the pipeline did. That's fine until the finding is unfavourable, at which point the independence question surfaces immediately.
It's unrepeatable at scale. One metric in a week is survivable. Twelve metrics across four reporting periods is a project with a headcount request attached, and the request arrives at the worst possible time.
And it degrades. Every quarter that passes makes the reconstruction harder, because the people who remember the change leave, the Slack workspace gets archived, and the screenshots stop being findable.
The cost curve runs the wrong way too. Reconstruction is cheapest the first time it's requested and gets more expensive with every repeat, while evidence costs something to establish once and approaches zero per request afterward.
The evidence answer holds the three properties reconstruction lacks. It is independent, repeatable, and durable.
What an evidence answer actually requires
Four properties, and a platform either has them or it produces reconstruction with extra steps.
Attribution. Every definition change carries the identity of the human who decided it, distinct from the service account that wrote the record.
Prior state, retrievable in full. The question is what the definition said before, expressed the same way it's expressed now. A change log that records "definition updated" with a timestamp answers when and leaves what unanswered, which is the part being asked about.
Scope known at the time of change. Which reports, models, and downstream systems consumed that definition on the day it moved. Reconstructing consumption after the fact is harder than reconstructing the definition itself, because the consuming systems have also changed since.
Environment separation. Evidence that a change was made and reviewed somewhere before it reached production reporting. This is what distinguishes a controlled change from an edit that happened to be logged.
Those four are the difference between a change record and an audit trail. Many platforms produce the first and describe it as the second.
Drift is the same problem on a longer timescale
Audit makes this acute, and it's the same mechanism that handles the quieter version.
Definitions move gradually. A filter gets added, an owner leaves and their successor interprets a rule differently, a source system is replaced and the new field is close enough. No single change is remarkable. The cumulative effect is that the metric reported in 2026 measures something different from the metric reported in 2024, and the trend line drawn between them is wrong.
An organization with a diff catches that in a comparison. An organization without one catches it when an executive asks why the trend moved, which is the same reconstruction project arriving through a different door.
Regulated reporting regimes have been converging on this expectation for years. Whichever ones apply to your organization, the common demand is traceability with support, and the support is the part that gets improvised.
What to test when you evaluate
Bring one of your own definitions to any platform evaluation and ask for four things in the demo, in this order.
- Show me what this definition said twelve months ago, in full.
- Show me the person who changed it, and when.
- Show me which reports consumed it at that time.
- Show me the same change moving from a lower environment into production, with the review step visible.
The first question sorts the field faster than any feature comparison. A platform that stores current state alongside a change log will answer it partially and confidently, and the gap becomes visible within about a minute. Ask it early in every evaluation.
All four are answerable by demonstration, so ask them of everyone you evaluate. The Technology layer of this Kit shows how one platform answers them, in the same order, so you can hold it to the same test.
