Skip to content
$11.95 off at print Pre-order list →

Chapter 06

When AI Sees But Doesn’t Understand

Chapter opening

Three-Body Consciousness: Perception, Processing, Awareness

The expert perspectives in this chapter are drawn from synthesized interviews—detailed conversations constructed from their published work, research, and documented ideas. While the quotes reflect their established positions and frameworks, these are not transcripts of conducted interviews.

When AI Sees But Doesn’t Understand

October 21, 2018. Tempe, Arizona. A self-driving Uber killed Elaine Herzberg—the first pedestrian death caused by an autonomous vehicle.

The car’s sensors detected her 5.6 seconds before impact. The AI classified her as “unknown object,” then “vehicle,” then “bicycle,” then “unknown object” again. It never classified her as “pedestrian.”

The car saw her. It didn’t understand what it was seeing.

This is the difference between pattern recognition and consciousness. The AI had perception (sensors detected her) and processing (algorithms analyzed the data), but it lacked the third body: awareness of what the coordination between perception and processing actually meant.

The Consciousness Three-Body System:

  • Perception (sensing reality)
  • Processing (analyzing patterns)
  • Awareness (understanding what it means)

Two bodies can recognize patterns. Three bodies create consciousness.


The National Transportation Safety Board investigation peeled back layers of technical and organizational failure. Herzberg was walking her bicycle across a four-lane road outside a crosswalk, in the dark. The vehicle’s lidar, radar, and cameras detected her 5.6 seconds before impact but kept reclassifying her—unknown object at 5.6 seconds, vehicle at 4.0 seconds, bicycle at 3.3 seconds, unknown object again at 2.7 seconds. Uber’s software included a “suppression zone” designed to smooth out erratic readings and prevent false positives. Combined with repeated misclassification, the system never initiated emergency braking. At 1.5 seconds before impact, the system determined a collision was imminent but the suppression zone prevented action. Uber had disabled the Volvo’s factory-installed automatic emergency braking system to prevent “erratic vehicle behavior” during testing. The system relied solely on its own algorithms—algorithms that had just spent five seconds cycling through wrong answers. Impact at 43 mph.

The safety driver, Rafaela Vasquez, was streaming a TV show on her phone for 6.7 seconds before the crash. Both the AI’s limited awareness and the human’s distracted awareness failed to coordinate, leading to a fatal outcome.

Norbert Wiener would have recognized this failure instantly. The father of cybernetics built anti-aircraft predictors during World War II that faced the same structural problem: how do you coordinate a system’s response with a target that keeps changing? A bomber weaving overhead, a gun on the ground, and several seconds of shell travel time between them. If the bomber flew straight, the mathematics would be trivial. But the pilot maneuvers deliberately, introducing uncertainty. Wiener’s breakthrough was realizing that the pilot, despite maneuvering, exhibited statistical regularities—reaction times, banking angles, evasive patterns that revealed structure within the uncertainty. Not deterministic structure. Stochastic structure.

Wiener’s solution was feedback—the third body that turns blind shooting into adaptive coordination. The gun, the plane, and the feedback loop between them formed a cybernetic system. Remove the feedback and the gun fires at where the plane was. It misses. It fires again at where the plane was. It misses again. No learning occurs. No adaptation. No intelligence. Add the feedback—observe the error, adjust, fire again, observe the smaller error—and the system transforms. It coordinates with a moving, purposeful, uncertain reality. The gun does not control the pilot. It cannot control the pilot. It coordinates with the pilot’s statistical behavior.

Wiener named this coordination architecture “cybernetics”—from the Greek kybernetes, the steersman who coordinates ship, sea, and destination. The steersman does not optimize the ship toward the harbor while ignoring the wind. A steersman who ignores the wind is not bold. He is dead. Wiener spent the next sixteen years warning that systems optimizing without feedback from their context would achieve exactly what they were told and destroy everything they ignored. The Uber system was precisely such a system: optimizing classification without coordinating with the meaning of what it classified. It had perception and processing but no feedback arc closing through awareness. Two bodies cycling. The third body absent.


Antonio Damasio, the neuroscientist who overturned three centuries of Western philosophy about minds and bodies, offers a deeper diagnosis of why. His research on patients with ventromedial prefrontal damage reveals that reasoning without feeling is not pure reason—it is broken reason.

Chapter 06 · 3,924 words left · about 20 min in full

Keep reading. It's free.

One email unlocks every chapter in full, including the capstone. Enter the same email on any other device to keep reading there. No paywall, no tracking.

4,630 words in Chapter 06. Your email stays with the author and is never sold or shared.

Causal diagram: The Comprehension Gap

P Perception
→
R Processing
→
A Awareness
LATENT / Comprehension — the missing third body

The Uber system had perception and processing — but not the third body: awareness grounded in consequence. Two-body systems can see. They cannot understand what they see.

CAUSAL GRAPH · THE COMPREHENSION GAP

Nine Voices · Chapter 06

Consciousness as Pattern Recognition

46

Stuart Russell

AI Safety, UC Berkeley

The problem is not that we are building intelligent machines. The problem is that we are telling them exactly what we want. A machine that pursues a fixed objective does not coordinate with human values — it optimizes over them.

Synthesized for Knowware from Stuart Russell's published work. Not a quote. knowware.systems/method

Academic
47

Dr. Timnit Gebru

AI Justice, DAIR

A language model that predicts the next word is not understanding. It is coordinating with statistical patterns in text written by people who were never asked if they consented. That coordination is not neutral. It encodes who was counted and who was not.

Synthesized for Knowware from Dr. Timnit Gebru's published work. Not a quote. knowware.systems/method

Academic
48

Kate Crawford

AI Now Institute

Every AI system has a body — it is just hidden. It is the mine, the server farm, the warehouse worker, the moderator. When you ask what AI coordinates with, the honest answer includes all of them.

Synthesized for Knowware from Kate Crawford's published work. Not a quote. knowware.systems/method

Academic
49

Norbert Wiener

Father of Cybernetics

The thermostat coordinates with the temperature. The dictator tries to control it. The thermostat is wiser than the dictator. This is mathematics.

Synthesized for Knowware from Norbert Wiener's published work. Not a quote. knowware.systems/method

Practitioner
50

Margaret Mitchell

Hugging Face Ethics

If you don't know what a model was trained on, who it was trained for, and what it fails at — you don't know the model. You only know its output. And output without context is the definition of a stochastic parrot.

Synthesized for Knowware from Margaret Mitchell's published work. Not a quote. knowware.systems/method

Practitioner
51

In-Q-Tel Operator

Intel community ↔ tech

The question we ask about every technology is not: what does it do? The question is: what does it see? Pattern recognition at scale is not intelligence. It is the precondition for coordination — or for its denial.

Composite voice, synthesized for Knowware from the public record. Not a real person or a quote. knowware.systems/method

Practitioner
52

Sir Roger Penrose

Consciousness & Quantum

Gödel shows that human mathematical understanding exceeds any algorithm. The question is what that something is. I believe it's quantum.

Synthesized for Knowware from Sir Roger Penrose's published work. Not a quote. knowware.systems/method

Visionary
53

Antonio Damasio

Somatic Markers

The brain does not make decisions alone. The body is always in the room. It has already voted — long before the frontal lobe begins its deliberation. Consciousness is not the general. Consciousness is the diplomat who arrives after the negotiation.

Synthesized for Knowware from Antonio Damasio's published work. Not a quote. knowware.systems/method

Visionary
54

Rupert Sheldrake

Morphic Resonance

Why does a crystal know what shape to take? Why does a flock of birds turn as one? There is a field between things — a coordination medium — that carries the memory of form. We call this field morphic. Science calls it impossible. Nature calls it Tuesday.

Synthesized for Knowware from Rupert Sheldrake's published work. Not a quote. knowware.systems/method

Visionary