The Judgment Crisis: Human Decision-Making in the Age of AI

We will not lose judgment because machines take it from us. We will lose it because delegation is easier than deciding.

The last crisis was about understanding data. This one is about something the machines just learned to imitate. Judgment is now the scarce resource, and most of us handed it over without noticing.

A couple years ago I wrote that those who cannot think with data will be ruled by those who do. That article, The Data Literacy Crisis, became one of the most read articles that I have published. Readers recognized the problem because they were living it. Too much data, too little understanding, and a growing gap between the people who could interpret the numbers and the people governed by them.

The warning was right, but it is also already out of date. When I wrote it, I was working from an assumption I did not spend much time questioning. The data still needed a human to interpret it. Someone had to read the chart, question the source, and turn the numbers into a decision. Data literacy mattered because interpretation was still something we had to do.

That is what has changed. The machine now supplies the interpretation too. It reads the chart, writes the summary, drafts the recommendation, and delivers all of it so smoothly that it passes the only test most of us ever give to it: Does it make sense when I read it? The crisis did not go away. The problem moved from understanding the data to judging the answer.

A Sixty-Year-Old Warning

People have been warning about versions of this problem for a long time. In 1960, Norbert Wiener warned that before we hand our purposes to a machine, we need to be sure the purpose we give it is actually the one we want. In 1976, Joseph Weizenbaum made an important distinction. Computers can decide, but only humans can choose, because choosing involves values and responsibility, not just calculation. Carl Sagan and E.O. Wilson later raised a related concern: we were becoming very good at producing information without necessarily getting better at knowing what to do with it.

What strikes me about those warnings now is that they all assumed a person would still be doing the interpretation. No matter how much information we produced, someone still had to make sense of it. We had to read it, weigh it, decide what we believed, and figure out what to do next. That part was still ours. That assumption held for sixty years but it no longer does.

People have been warning about pieces of this problem for decades

Norbert Wiener worried about machines pursuing purposes we did not actually intend. Joseph Weizenbaum argued that because computers cannot be wise, we should not give them tasks that require wisdom. Carl Sagan warned of a society increasingly unable to distinguish what feels good from what is true. E.O. Wilson put the shortage plainly: “We are drowning in information, while starving for wisdom.”

Each saw a different part of the problem. None imagined a machine that could do the sense-making itself.

The Handoff Nobody Felt

A hundred times a day, something makes or shapes a decision for you, often without you thinking of it as a decision.

1. A model ranks the job applicants before a human reads a single resume
2. A feed chooses what your kid will believe is normal tonight
3. A score sets your neighbor's interest rate
4. A dashboard tells your team what is working, and the number gets trusted more than the people who watched the work happen
5. A chatbot answers your customer in your company's name while you are in a meeting

Most of these do not feel like decisions, though. They show up as things like a score, a recommendation, a ranking, or a default. We use them because that is what they are there for. And most of the time, we probably should. The problem is that it becomes very easy to stop noticing where the system's input ends and our judgment begins.

I have spent the last few years looking at what happens when that goes wrong. For example, when a man was arrested in his own driveway, in front of his daughters, because a facial recognition lead got treated as a verdict. Or when a tenant with sixteen years of on-time rent was denied an apartment by a screening score nobody could explain, and nobody could appeal. Or when a lawyer with thirty years of experience filed a federal brief citing six cases that never existed, because he trusted the answers AI gave him.

These cases are very different, but there is something important they all share in common. The system provided an answer, score, target, or recommendation. A person or organization still had to decide what to do with it. Somewhere along the way, that distinction disappeared.

You Are on Both Sides of the Handoff

The data literacy conversation mostly focused on helping people understand the data in front of them. But this problem is different, because we are not just receiving the outputs of these systems. We are also deciding how they get used.

At home, the distinction is easier to see. You are the parent whose teenager is being shaped by a feed that has learned exactly what holds their attention, which is rarely the same as what is good for them. You might be the applicant a model ranks before a human sees your name, or the patient, borrower, or customer whose options are shaped by a score you may never see or understand.

Then you go to work, and the roles change. We build these systems, buy them, set the targets they optimize for, and decide how much authority to give their outputs. A ranking influences what people see and believe or a metric on a dashboard can quickly become the thing people work to improve, even when it is only a proxy for what we actually care about.

I don't think we connect those two experiences often enough. We worry about how algorithms influence us and the people we care about, while also making decisions about how algorithms will influence someone else. That is part of what makes this judgment crisis different. It is not something happening to us. We are doing it to each other, mostly without malice and mostly without noticing.

The New Scarcity

Looking back at The Data Literacy Crisis, I think I would frame the problem differently today.

The information age made attention the scarce resource. The AI age makes judgment the scarce resource. We are no longer just drowning in data and starving for wisdom. We are drowning in answers, and starving for the ability to tell which answers deserve our trust, and which decisions were ours to keep.

That does not mean we should trust machines less. In many situations, a model can be more consistent and more accurate than a person. Decades of research have shown that human experts are not nearly as good at some kinds of judgment as we like to think. Sometimes giving more of the work to a machine is exactly the right decision.

The harder question is knowing when. We can trust the machine when we should be questioning it. We can also reject it when the evidence tells us it performs better than we do. The important decision is not simply whether to use AI. It is deciding how much authority to give it in a particular situation.

That is what I mean by judgment. Judgment is not the opposite of automation, and it does not mean keeping a human involved in every decision. It is deciding what should be automated, what should be trusted, what needs to be checked, and what responsibility still belongs to us.

Think about teaching a teenager to drive. You do not decide one morning that you either trust them or you don't. You let them drive around an empty parking lot before you send them onto a busy highway. As they show you what they can handle, you give them more responsibility. And how much evidence you need before doing that depends on what could happen if they get it wrong.

We should think about AI in much the same way. The question is not whether we trust it. The question is what we trust it to do, under what conditions, and what happens when it is wrong.

Why We Hand It Over

So, if handing over judgment can create these problems, why are we so willing to do it?

Part of the answer is simple. Humans have always found ways to offload some of our thinking. We write things down so we do not have to remember everything. We use rules of thumb because we cannot weigh every option from scratch. We rely on specialists because none of us can know or check everything ourselves. There is nothing inherently wrong with that. We have limited time and attention, and offloading some of the work lets us use both somewhere else.

But effort is only part of it. Decisions also come with uncertainty and responsibility. We might not know the right answer or we might make the wrong call. And if we do, we may have to explain why.

Research suggests those things affect our willingness to delegate. We are more likely to hand off decisions when the outcome could be unpleasant or when we would rather not own the consequences. Shared responsibility can also change how closely we examine a decision. Delegation does not just reduce the amount of work we have to do. It can change how responsible we feel for the answer.

That is what makes AI so interesting. It can relieve several of those pressures at the same time. It can do some of the thinking, give us an answer when we are uncertain, and make the answer feel like it came from somewhere other than us.

A calculator can do the arithmetic, but you still have to decide what problem you are solving. Writing something down helps you remember it, but you still have to decide what it means. AI can go much further. It can interpret the information, explain what it thinks is happening, recommend what to do, and give you a finished answer that sounds confident and complete.

That makes delegation very easy.

Delegation can reduce the effort of thinking, the discomfort of uncertainty, and even how responsible we feel for the answer. But feeling less responsible does not necessarily make us less responsible.

Read the research behind this →

Where This Is Already Starting To Show Up

When I wrote The Data Literacy Crisis, I ended with some predictions about where I thought we were heading. I don't think we have to predict anymore. We can already see some of the problems taking shape.

One of the ones I worry about most is what happens to people who are learning a job while AI can already do parts of that job for them. There is a lot of work early in a career that is slow, repetitive, and sometimes frustrating. It is also how you learn. You make mistakes and you learn what good looks like and, just as importantly, what doesn't look right.

Think about learning to drive with GPS. GPS can get a new driver where they need to go from day one. But if it always tells them where to turn, they may get to the destination without ever really learning the roads. The immediate result is better. They get there faster and make fewer wrong turns. But take the GPS away, or let it give them a bad direction, and you find out what they actually learned.

AI creates a similar risk at work. People may produce better work sooner while missing some of the experiences that help them develop their own judgment. The concern is not that they used the tool. It is whether they learned enough along the way to know when the tool is wrong.

We are also starting to wrestle with responsibility. Using a model, score, or recommendation does not remove the need for someone to decide whether it should be trusted. If an organization uses a system to make or influence a decision, someone still has to be accountable for how that system was used. Saying "the model said so" does not answer the question of why we trusted it.

Then there is the problem of truth. AI can produce an answer that is clear, confident, and completely wrong. That matters because clarity and confidence are signals we have always used when deciding whether something makes sense. Now we have a technology that is exceptionally good at producing those signals. The answer can feel right before we have done anything to determine whether it actually is.

Organizations also have their own version of this problem. We have spent years building dashboards, metrics, targets, and now AI systems to help people make better decisions. But a system can give you an accurate answer to the wrong question. A team can hit the target while missing the outcome the target was supposed to represent. For example, a dashboard can be green while the customer is still having the same problem.

None of those are really technology failures. They are judgment failures. The technology can give us an answer. We still have to decide what the answer means, how much to trust it, and what to do with it.

What Cannot Be Delegated

It would be easy to end here by saying that machines can provide the answers, but humans still have to provide the judgment. Unfortunately, I don't think it is that simple anymore.

AI can already evaluate its own output, compare alternatives, apply decision rules, estimate confidence, and flag situations that need more attention. And those capabilities will continue to improve. So I don't think judgment itself is something we can simply declare to be uniquely human. The more important question is who decides how much judgment to hand over.

AI can help us decide where automation makes sense. It can tell us where it performs better than people, where it is uncertain, and where additional review might be useful. But it cannot make the decision about what responsibility we are willing to give up. That is still a choice someone has to make and be accountable for.

And that is where delegation gets complicated. Giving a decision to a system can make us feel less responsible for the outcome. But feeling less responsible does not necessarily make us less responsible. If your company uses a model to screen an applicant, approve a loan, recommend a treatment, or answer a customer, someone made the decision to use that system and decided how much authority to give it.

The machine may have produced the answer. Someone still decided that the answer was good enough to act on.

That is why I don't think the next challenge replaces data literacy. We still need people who can read the numbers, question the evidence, understand what a model is telling them, and recognize when something does not look right. But that is no longer enough. We also need to get much better at deciding when to trust the answer, when to question it, when to let the machine decide, and when the decision still needs to be ours.

And underneath all of those questions is an older and harder one. Who decides?