The Day We Stopped Checking

There's an old question that doesn't get asked enough anymore: How do you know that?

Not as a challenge. Just as an honest inquiry. How do you know what you know?

Think about it for a moment. When you learned that the Battle of Hastings was fought in 1066, you didn't travel back in time to verify it. When you understood that the Earth orbits the Sun, you didn't break out your calculator and do the math yourself. You trusted someone. A teacher, a textbook, or maybe even an encyclopedia. And that person trusted someone before them, who trusted someone before them, going back and back until you reach the people who were actually there, or who actually did the math.

That's not a shortcoming in how we think. That's just the reality of being human. Our personal experience is a tiny island. The ocean around it is everything we've had to take on faith from someone else.

Philosophers have a word for those two sources of knowledge: experience and authority. We gather what our own senses and life can teach us, and then we fill in the rest — the overwhelming, vast majority of the rest — from authority. From people and institutions we've decided to trust.

And for a long time, that worked pretty well.

Garbage In

Most of us have a fairly good working instinct for how to sort true from false. First, you gather your facts, from whatever sources you have. Then you apply a few self-evident principles, like: if A equals C, and B equals C, A and B probably equal each other. And then you arrange those facts into a coherent picture, one piece connecting to the next, until the weight of it all points somewhere.

It's not a perfect process. But it's a surprisingly reliable one, when the inputs are good.

That's the catch. The whole structure depends on the quality of what goes in. Garbage in, garbage out, as they say. If the facts you're working with are wrong, even flawless reasoning leads you straight off a cliff. You can be completely logical and completely mistaken at the same time, and never know it.

This is why the corruption of authority is such a serious problem. Not just politically or socially, but epistemically, at the level of how we form beliefs and make decisions.

When an institution that people rely on for facts starts distorting those facts, the damage doesn't stop with the people who hear it first. It travels. It gets passed along, cited, built upon. By the time anyone thinks to question it, it's already three or four layers deep in the chain. The original distortion is buried under a pile of legitimate-looking references, each one confidently pointing back to the one before it. And everyone, at every step, did exactly what they were supposed to do. They trusted authority.

No Bottom to Tap

Here's where it gets interesting.

Artificial intelligence has entered this chain. And it has done so with a wrinkle nobody has quite figured out how to iron out.

Every fact a human being holds can, at least in principle, be traced back to something experienced. Not necessarily by you. But by someone. A sailor who crossed that ocean. A scientist who ran that experiment. A journalist who was in the room. At the far end of every chain of authority, if you follow it far enough, there is a person who saw something with their own eyes.

AI doesn't have that. It has no eyes. No experiences. No moments of standing somewhere and observing something directly. What it has is text, enormous amounts of text, produced by humans who did have experiences, or who were themselves relying on other humans who did.

That matters more than it might seem. Because it means that AI is operating entirely within the chain of authority, with no independent access to the bedrock of experience underneath it. It can only tell you what it has read. And what it has read was written by people who were themselves, in large part, relying on what they had read.

So when you ask AI something and it gives you a confident answer, you're getting the distilled output of a very long and very complex game of telephone. Most of the time, that output is reasonable and useful. But the question of how you'd verify it is genuinely difficult. Because there's no bottom to tap. No original experience to appeal to.

The Most Articulate Authority Yet

Of course none of this means AI is useless, or that we should treat it with suspicion in the way we might treat a known liar. That would miss the point.

The point is something more like: we are being asked to add a new kind of authority to our chain, and we haven't fully thought through what that means.

Every generation has had to navigate the expansion of authority. When printing became widespread, people had to learn to evaluate books. When mass media arrived, they had to learn to evaluate broadcasts. When the internet came along, they had to learn to evaluate websites. Each time, the volume of available "authority" exploded faster than the tools for evaluating it could keep up. And each time, the people who adapted best were the ones who kept asking: How do you know that?

We're at one of those moments again. Except this time the authority in question is more articulate, more confident, more fluent, and more persuasive than any previous technology. It doesn't hedge the way an uncertain person hedges. It doesn't visibly sweat or fidget when it's out of its depth. It just answers.

That smoothness, while useful, is also a little dangerous. Not because the technology is acting in bad faith, but because our instincts for detecting unreliable authority were built around human cues like hesitation, inconsistency, or the look on someone's face when they're making something up. Those cues don't translate.

What Atrophies in the Convenience

The deeper concern isn't really about AI at all. It's about us.

Reasoning is a skill. Like any skill, it atrophies without use. The habit of checking, questioning, and following a claim back toward its source, that takes effort, and over time, it takes less effort if you do it regularly and more effort if you don't.

There's real convenience in having a confident, articulate source that can answer almost any question instantly. And convenience has a way of slowing reorganizing our habits. You stop doing the thing you no longer need to do. And then, after a while, you find you've also lost the ability.

A population that outsources its reasoning doesn't just become dependent. It becomes vulnerable. Vulnerable to whoever shapes the authority it's relying on. Vulnerable to errors that compound invisibly across layers of citation. Vulnerable, in the most fundamental sense, to not being able to tell the difference between true and false.

That's not a technology problem. That's a human problem with a very long history. The technology just makes it more urgent.

Still in the Game

So what do you do with all this?

The same thing people have always done, when the world got more complicated: you go back to the basics and you hold them more deliberately.

You keep asking how people know what they claim to know. You stay curious about the difference between something experienced and something reported. You remember that confidence and correctness are not the same thing, that fluency is not a substitute for accuracy, and that the length of a chain of authority says nothing about whether the first link was sound.

You use the new tools, because they are genuinely useful. And you don't let them use you.

The question How do you know that? is not a sign of suspicion or stubbornness. It's a sign that you're still in the game, still thinking for yourself, still doing the work that nobody, and no technology, can do entirely in your place.

Keep asking it.

---

To learn more about Living Systems Leadership, schedule a free call today.