Data quality is a property of the data. Evidence fitness is a relationship between the data, the question, and the cost of being wrong. Stewardship has mastered the first. As AI systems move from displaying data to recommending action, organizations need the second.
AI did not end the data literacy crisis. It changed it. As machines move from providing information to interpreting it and recommending what to do, the scarce skill is becoming judgment: knowing what to trust, what to question, and what decisions are still ours to make.
Evidence does not make decisions. People interpret it, weigh it, challenge it, and make judgments. A courtroom reveals what leaders can learn about evidence, expertise, confidence, and data-informed decision-making.
Data quality is a property of the data. Evidence fitness is a relationship between the data, the question, and the cost of being wrong. Stewardship has mastered the first. As AI systems move from displaying data to recommending action, organizations need the second.
We have become remarkably good at proving our data is right. We are far less disciplined about proving our conclusions deserve to be believed.
Organizations have spent years making their data accurate, consistent, and trustworthy. That work matters, and it stops at the dataset. A metric can pass every quality check and still be the wrong evidence for a consequential decision. I call the missing standard evidence fitness: whether the whole argument holds, including the data, the reasoning applied to it, the explanations left unexamined, and the cost of being wrong. As AI moves from showing data to explaining it and recommending action, the distance between what the evidence establishes and what the system suggests keeps growing.
If you lead, you have probably made a call on a dashboard number you never questioned, because it came certified. If you work in data, you have probably certified a dataset and never seen the decision it fed. Each handoff meets its obligation. The reasoning that connects them often belongs to no one, and that is where good data becomes a bad decision.
Data stewards and governance leads wondering where their responsibility ends. Analysts whose insights travel further than their caveats. Executives who sign off on decisions built from dashboards and, increasingly, from AI recommendations. Chief data officers deciding how governance has to change once systems start generating explanations of their own.
A clear line between data quality and evidence fitness, plus four questions you can bring to your next meeting: relevance, coverage, inference, and consequences. Three practical shifts that connect stewardship to decision quality without making stewards responsible for everything. And a simple way to find the unowned gap in your own organization, the place where an accurate metric turns into a consequential decision and no one checks the reasoning in between.
Imagine a financial institution reviewing branch performance. The dashboard shows a clear trend: in-person transactions have declined 30% over three years.
The numbers are accurate. The calculations have been validated. The definition of a transaction is consistent across branches. The data is current, complete, and traceable to its source. From a data stewardship perspective, everything is working.
Leadership concludes that customers are moving to digital channels. With transaction volumes falling, the branch is hard to justify. They close it.
Six months later, the consequences become apparent. The branch had served a disproportionate number of older customers who relied on in-person service. It had also been the place where relationships with small businesses got started. Transaction volume had declined, but the branch was still contributing to retention and relationships that the transaction metric never captured.
Some customers move to other branches. Others leave the institution entirely.
The data was right. The decision was wrong.
The example is hypothetical, but the failure is familiar. Organizations invest heavily in making their data accurate, consistent, and trustworthy. Those investments cannot guarantee that people will interpret the data appropriately or decide well with it. The problem begins after the data has been deemed trustworthy.
That raises a question for stewardship. If we have governed the data well, and it is then misinterpreted or used to justify a poor decision, has stewardship done everything it needs to do?
Stewards establish and maintain the conditions that make data usable and trustworthy. Their work covers data quality, business definitions, metadata, lineage, access, and the consistent application of governance policy.
These activities solve real problems. Without them, two departments calculate customer retention differently. An analyst combines incompatible datasets without knowing it. A business leader relies on stale information. Good stewardship reduces all of these risks.
What traditional stewardship activities share is a focus on the condition and management of the data itself. Is it accurate? Complete? Consistently defined? Traceable? Protected?
Those questions are necessary. They do not address what happens when someone uses the data to form a conclusion.
A dataset can satisfy every quality requirement and still be unsuitable for a particular decision. A dashboard can display accurate information while encouraging a misleading interpretation. A model can run on well-governed data and produce a recommendation that overlooks important consequences.
None of this diminishes traditional stewardship. It exposes the limits of what traditional stewardship alone can accomplish. Trustworthy data and trustworthy decisions are different things.
A dashboard can display accurate information while encouraging a misleading interpretation.
I use a framework called DIKUW: Data, Information, Knowledge, Understanding, and Wisdom. These are not five sequential steps so much as five interconnected ways we observe, interpret, explain, and respond to the world.
The first two are closely related. Data consists of recorded observations. Information emerges when data is organized and placed in context so it communicates meaning. A dashboard showing 1,200 transactions provides data. Knowing those transactions represent a 30% decline from three years earlier provides information.
Knowledge and understanding are where sense-making happens. Knowledge lets us recognize patterns and principles drawn from education, experience, and analysis. Understanding goes further. It means grasping why things happen and how factors interact. Knowing that branch transactions are declining is different from understanding why. Perhaps customers are adopting digital banking. Perhaps the branch cut its hours. Perhaps the local population changed. Perhaps transaction volume is down while the value of advisory conversations is up. Each explanation points to a different decision.
Wisdom is the exercise of judgment about what should be done, weighing the evidence, the uncertainty, the values at stake, the alternatives, and the consequences. Even if leadership fully understands why transactions are declining, that understanding does not settle whether the branch should close. The decision still requires judgment.
This is where data quality reaches its limit. Stewardship can ensure the transaction counts are accurate and their meaning is clear. It cannot establish whether transaction volume is sufficient evidence to justify closing a branch. That is a different question, and it deserves far more attention than it gets.
We talk about data quality as though it were an inherent property of a dataset. To some extent it is. We can check whether records are complete, values are accurate, and definitions are consistent.
But the usefulness of data also depends on what someone intends to do with it.
The branch transaction data shows how this works. If the question is “How many transactions occurred last month?” the dataset is perfectly suitable. If the question is “How should we adjust staffing?” it is useful, especially combined with transaction timing and service requirements. If the question is “Should we close this branch?” transaction volume alone is unlikely to be enough.
The data hasn’t changed. The decision has.
I call this distinction evidence fitness. Evidence fitness asks whether the available evidence is appropriate and sufficient for a particular question or decision. It considers the quality of the underlying data, but also its relevance, its representativeness, its limitations, and the strength of the reasoning that connects it to a conclusion. Most importantly, it considers the consequences of being wrong.
A modest evidence base might justify experimenting with staffing schedules. Closing a branch, with significant and hard-to-reverse consequences, warrants a much stronger one.
The principle is simple to state. Data quality is a property of the data. Evidence fitness is a relationship between the data, the question, and the consequences of the decision.
Data quality practitioners will point out that their field already defines quality as fitness for purpose. That’s true, and it’s the right standard for the asset. But fitness for purpose asks whether a dataset suits a task. Evidence fitness asks whether the whole argument holds: the data, the reasoning applied to it, the assumptions underneath, the explanations left unexamined, and the uncertainty that remains. A dataset can be fit for purpose and still carry an argument it cannot support.
A dataset can be fit for purpose and still carry an argument it cannot support.
Four questions get at it. Relevance: does the evidence address the decision we’re making, or a nearby one? Coverage: what populations, dimensions, or explanations does it leave out? Inference: how much reasoning sits between the evidence and the conclusion? Consequences: is the strength of the evidence proportional to the uncertainty, reversibility, and impact of the decision?
A steward can reasonably certify that a dataset meets quality standards. Those four questions cannot be answered at certification, because they depend on a decision that hasn’t been made yet. They need business knowledge, analytical reasoning, and judgment, applied at the moment the evidence is used.
For most of their history, dashboards have been information products. They organize data, show trends, highlight exceptions, and let people monitor performance. The user interprets what the dashboard shows and decides what to do.
That division of labor is changing. Modern platforms offer automated insights, natural-language explanations, predictive models, and recommended actions. Generative AI is accelerating the shift. Instead of simply displaying a decline in retention, a system might identify associated patterns, propose explanations, and recommend interventions.
This creates real opportunity. It also introduces new risk.
Picture a dashboard that presents the following progression:
The observation may be accurate. The pattern may be supported by the data. But the explanation introduces a causal claim that has not been established. Perhaps customers who are already disengaging simply use digital services less. Perhaps some third factor drives both digital engagement and retention. The recommendation then builds on an explanation that may be wrong.
Even if the explanation were right, and poor digital onboarding really does drive attrition, the recommendation still hasn’t earned its place. Knowing why something happens does not tell you what to do about it. A different intervention might cost less, reach more customers, or carry fewer side effects. The explanation is a question of understanding. The recommendation is a question of judgment.
Knowing why something happens does not tell you what to do about it.
Each step moves further from the original observations and adds assumptions. Yet the system presents the whole progression in one consistent, authoritative voice. To the user, the line between an observed fact, an inferred relationship, a proposed explanation, and a recommended action may be invisible.
The more capable our systems become at generating interpretations, the more important it becomes to distinguish what the evidence establishes from what the system suggests.
Historically we have invested in documenting where data came from, what it means, and whether it can be trusted. As systems increasingly generate explanations and recommendations, we also need ways to make the reasoning that follows from the data visible and open to examination. A well-governed dataset does not guarantee a well-supported explanation. A well-supported explanation does not guarantee a wise recommendation.
Does this mean data stewards should become responsible for every interpretation, recommendation, and decision involving organizational data?
No. That would be unrealistic and counterproductive. Stewards already operate with limited authority and competing expectations. Holding them accountable for every downstream decision would create responsibility without control.
The better answer is to recognize that responsibility for evidence must stay connected as it moves from data toward decisions. That means distinguishing two complementary concerns.
Stewardship of the asset. Traditional stewardship ensures data is trustworthy, consistently defined, appropriately managed, and accessible to authorized users. These responsibilities remain essential. Without them, downstream reasoning starts on unstable ground.
Responsibility for evidence use. A complementary set of responsibilities covers how data is interpreted and used to support conclusions: evaluating whether evidence suits the intended purpose, separating observations from explanations, communicating uncertainty, examining assumptions, and considering alternative interpretations.
These responsibilities cannot belong to stewards alone. Analysts own their methods and the validity of their conclusions. Business experts contribute the domain knowledge needed to interpret patterns and test explanations. Decision-makers remain accountable for weighing alternatives and choosing. Governance leaders set the expectations that tie these roles together.
The change required is not a new role or a longer job description for every steward. The organization needs mechanisms for what happens after trustworthy data becomes evidence in a decision.
Without those mechanisms, accountability fragments. The data team certifies the dataset. The analytics team produces an insight. A business leader acts on the recommendation. Each group has met its obligation, and the argument connecting them belongs to no one.
How Accountability Fragments
Every handoff passes its check. The decision still goes wrong.
Who in your organization is expected to ask that last question?
This is the failure in the branch example. No one misread the transaction data. The organization had simply built a process with no checkpoint between an accurate metric and a consequential decision. We have spent two decades establishing who owns the quality of data. We have rarely established who owns the quality of the reasoning that follows from it.
We have spent two decades establishing who owns the quality of data. We have rarely established who owns the quality of the reasoning that follows from it.
If organizations want stewardship to contribute more directly to better decisions, I see three practical shifts.
Keep measuring and improving data quality. For consequential decisions, also ask whether the available evidence is fit for the intended purpose. That means making the decision itself part of the conversation. What question are we answering? What evidence would be relevant? What are its limitations? What happens if our conclusion is wrong? The answers should shape which data gets collected, how it is analyzed, and how confidently results are presented. The appropriate standard of evidence depends on the decision being made.
Traditional metadata explains what data represents. Knowing a metric’s definition is not always enough to interpret it appropriately.
A common definition of active customers is anyone who completed at least one qualifying transaction in the previous 90 days. That definition is useful. Additional context might note that the measure underrepresents customers who hold significant balances but transact rarely. The first piece explains how the metric is calculated. The second helps users see where it might lead them to an incomplete conclusion.
Every dataset does not need an exhaustive list of possible misreadings. Organizations should identify and communicate the limitations that are material to common or consequential uses, and put that context where people actually encounter the evidence.
Organizations need expectations for how important recommendations are developed and justified. For consequential decisions, that might mean documenting the evidence used, its limitations, the assumptions underlying the analysis, alternative explanations, and how confident decision-makers are in their conclusions. The four evidence fitness questions do most of this work; the organization only has to decide who answers them, and when, before a consequential decision is made.
It should also cover what the organization expects to happen next. What outcome would indicate success? What would challenge the assumptions? When should the decision be revisited? This lets the organization learn from the quality of its reasoning, not only from business results.
None of this should turn every decision into a bureaucratic exercise. A routine operational adjustment does not need the scrutiny of a branch closure or a major technology investment. Rigor should match the uncertainty, reversibility, and consequences of the decision.
This is where stewardship, data literacy, analytical capability, and decision governance reinforce one another. Stewardship supports the reliability and meaning of the evidence. Data literacy helps people interpret and evaluate it. Analytical capability investigates patterns and explanations. Decision governance sets expectations for how evidence informs consequential choices. No single discipline can do the job alone.
Dashboards are moving from telling us what changed, to helping us see why, to suggesting what we should do. That sophistication does not eliminate the need for human judgment. In some ways it raises the stakes. When systems produce information, users must interpret it. When systems produce explanations, users must evaluate them. When systems recommend actions, users must decide whether those actions are appropriate. Judgment does not disappear because a system can generate a convincing recommendation.
So the value of stewardship cannot be measured solely by whether the underlying data was accurate. We also need to ask whether people have the context, capabilities, and organizational support to use that data responsibly.
In the branch closure, the data was accurate. The dashboard was correct. The definitions were consistent. What failed was the reasoning that treated one measure of branch activity as sufficient evidence of the branch’s value.
We are becoming exceptionally good at collecting, processing, and presenting data. AI is extending those capabilities into interpretation, explanation, and recommendation. More sophisticated outputs do not automatically improve the quality of our judgment. If anything, they create a greater need to understand where evidence ends, where interpretation begins, and what remains uncertain.
Data stewardship has a central role in meeting that challenge. It cannot meet it alone.
The ultimate test of our investment in data is not whether the numbers were right. It is whether we used them wisely.
Pick one consequential decision your organization made in the last year on the strength of a dashboard. Write down the question it answered, the evidence it used, and what would have changed the conclusion. Then ask who, if anyone, was responsible for that last step. If the answer is no one, you’ve found where evidence fitness needs to live.
The three shifts in this article are a starting point. If you want to work through what evidence-use governance would look like for your data team, or bring this conversation to your stewards and analysts together, contact us. Members can also explore the decision tools in the TDIW Learning Hub, including the Decision Brief, which puts evidence fitness into a format decision-makers can sign off on.
