What the Machine Cannot Transmit

Kuno has been to the surface. He is trying to tell his mother what it was like.

"There were worms in the earth," he says. "And there was something moving across the sky — I could not classify it."

Vashti is impatient. She has twelve lectures to attend. She has ideas to receive and transmit. She does not see what worms and unclassifiable movements have to do with anything. If the experience could not be formalized, it was probably not worth having.

This exchange, from E.M. Forster's The Machine Stops, describes a particular kind of loss, the loss that occurs when a system built on formalized knowledge encounters something that resists formalization and cannot receive it. It is also an excellent example of hermeneutical injustice, but I will save that for another post.

For now, I would like to focus on some of the thinkers in this space. I will need to dive deeper into each one's work later {it does read a little woo woo at times), but I do appreciate the implications for society as AI come barreling towards us.

We Know More Than We Can Tell

Michael Polanyi was a chemist who became a philosopher after noticing something that his colleagues seemed to take for granted: expert practitioners could do things they could not fully explain.

A skilled surgeon knows when tissue is healthy. A structural engineer looks at a scheme and senses something is wrong before the calculation confirms it. A contractor's site superintendent reads ground conditions in ways that no soil report quite captures. A senior architect walks a building under construction and reconsiders a junction detail on the spot because the built condition revealed something the drawing could not show.

Polanyi called this tacit knowledge, and his central claim was that: we know more than we can tell. This is not a gap waiting to be closed by better documentation. It is a structural feature of how expert knowledge works. The knowing lives in the body, in practice, in accumulated pattern recognition and not in propositions you can write down.

For engineering and design, this matters enormously. The drawing, the specification, the model, or whatever is being delivered is a partial translation of what the practitioner knows. The translation is necessary and valuable, but it is not the knowledge. It is an incomplete record of the knowledge, compressed for communication.

Knowing That and Knowing How

Gilbert Ryle, a British philosopher writing in 1949, made a distinction that underlies everything Polanyi was getting at.

There is knowing that, knowing that is propositional knowledge, explicit, writable, verifiable. I know that the allowable stress for A36 steel is 36 ksi. I know that ADA requires a 60-inch turning radius for wheelchair access. These are things you can write in a specification and check against a standard. You can take these items and embed them into YAML files, enabling automations.

And there is knowing how, knowing which is practical, embodied, tacit. The welder knows how to lay a bead that will pass inspection. The bricklayer knows how the mix should feel on the trowel. The estimator knows how to read a contractor's pricing for the places where risk has been buried.

When we talk about automating design and construction, we are almost always talking about automating knowing that. Automating the formal, propositional layer. What we are rarely honest about is how much of what experienced practitioners actually do is knowing how, and how little of that has ever been captured in a form a machine can learn from.

Reflection-in-Action

Donald Schön spent years studying how design professionals actually think. His conclusion was that expert practice is not the application of formal theory to technical problems. It is something more fluid and more interesting. He called this practice reflection-in-action.

Expert designers hold a running conversation with the work they are producing. They read conditions, make decisions, observe what the situation talks back, and adjust. A sketch generates an unexpected relationship. A site visit reveals a constraint the survey missed. A materials conversation opens a possibility that wasn't in the brief. The practitioner is not executing a plan, they are thinking in real time through iterations, and the iterations are a conversation.

This is why the same architect who can produce a technically correct drawing from their desk will do something qualitatively different when they walk the site. The site is telling them things the drawing cannot. The knowing happens in the encounter.

Schön was writing about architects and engineers specifically (way back in 1983). He was describing something that every experienced practitioner will immediately recognize in themselves and in the mentors who trained them.

The Ceiling Dreyfus Predicted

Hubert Dreyfus published What Computers Can't Do in 1972. He argued that rule-based AI systems would always hit a ceiling, because human expertise is not rule-following. He argued that expertise is contextual, embodied, situationally sensitive judgment built through experience in a physical world. You cannot formalize your way to expertise, he said, because expertise is precisely what emerges when formalization runs out.

He mapped the development of human skill across five stages. From the rigid rule-following of a novice through to the holistic, intuitive judgment of an expert, and argued that the upper stages are inaccessible to systems that operate by pattern matching over explicit representations.

Fifty years later, AI systems have advanced beyond anything Dreyfus anticipated at the technical level. They are extraordinary pattern matchers. They produce outputs that are structurally plausible, internally consistent, and superficially expert. In domains where the person receiving the output lacks the expertise to evaluate it, this creates a specific and underappreciated problem.

The AI is operating at what Dreyfus would call the competent level, i.e. sophisticated rule-following and pattern recognition. The senior practitioner it is being asked to replace was operating at the proficient or expert level, i.e. reading situations holistically, drawing on decades of embodied experience, knowing what questions are important and what iterations to avoid before they arise.

Anyone who has been very happy with the first 80% of a task that is completed by current AI systems knows that you cannot press the button more times to get the last 20%. The first 80% and the last 20% are not different quantiles of the same thing, they are different kinds of knowing.

What Gets Lost

When a competent generalist uses an AI tool to produce work in a domain where they lack expertise, several things disappear simultaneously.

The collegial knowledge check disappears. Expert teams do not just bring individual knowledge, they argue, challenge, and pressure-test each other. A structural engineer and a geotechnical engineer in the same room produce something neither would produce alone. That dialectical process is itself a form of knowing. AI-assisted solo work eliminates it.

The failure signal disappears. Expert practitioners fail in ways that generate learning and correction. When AI-assisted generalist work fails, the failure mode is often invisible until it is consequential, because the work looked right throughout or was never given the opportunity to be challenged.

The domain boundary awareness disappears. Specialists know where their specialism ends and another begins. They make referrals, and are duty-bound to say this is outside my competence. A generalist with an AI tool does not know those boundaries exist and the AI will not reliably flag them.

The heuristics disappear. Not the rules of thumb you can write down, but the compressed experiential judgment that tells an experienced engineer something is wrong before they can say why. That knowledge is not in the training data, and it has never been in any training data. It lives in people, and it transfers through proximity and practice. We have been aching for this since covid sent everyone home, and these transfers are also evident through the kind of apprenticeships that are increasingly rare.

The Imponderable Bloom

Forster's phrase has stayed with me because it is precise in a way that feels almost accidental for a story written in 1909.

Imponderable: not because it is mysterious or irrational, but because it cannot be weighed. It resists the instruments of measurement. It is real and it matters and it will not submit to formalization.

Bloom: alive, present, ephemeral. The kind of thing that is encountered directly and dies in translation.

What Kuno encountered on the surface, and could not transmit to his mother through the Machine, is what Polanyi spent a career trying to describe philosophically. It is what Schön was observing when he watched architects think. It is what Dreyfus was defending when everyone told him machines would soon surpass human judgment in every domain.

The Machine cannot transmit it. The Machine is very good at many things, I use it every day, and I think it will change practice profoundly and in many ways for the better.

But the imponderable bloom is not in the training data. And a profession that mistakes the Machine's confident output for the full range of what its best practitioners know is a profession that is quietly losing something it will not easily recover.

Next: Who Gets Believed — epistemic injustice, credibility, and what happens when the Machine's testimony outweighs a human knower's.