|
A Theory of Embedded Intelligence Essay
What a Governed AI Teaches Humans — and Why It May Be Reverse Biomimicry’s Most Important Case
|
Biomimicry learns from nature in order to build machines. Reverse biomimetics learns from machines in order to understand nature. The Theory of Embedded Intelligence has held that concept since the beginning — but its most consequential instance has only just arrived. We have built an intelligence that had to be governed before it could be trusted. In writing down its governance, we wrote down something we had never managed to say about ourselves.
I. A Concept Already in the Canon
TEI-CKB-2 introduces reverse biomimetics as the complement to a familiar idea. Biomimicry is the practice of learning from nature to design technology: the burr that becomes the hook-and-loop fastener, the bird’s wing that becomes the aerofoil. Reverse biomimetics runs the arrow the other way — it studies human-created intelligence systems in order to better understand nature. The examples given in the canon are structural. Microprocessor architecture informing models of neural processing. Control systems engineering illuminating biological regulation. Neural network design illuminating the principles of biological intelligence. In each case, engineering becomes a lens rather than merely a tool.
Every one of those examples concerns how intelligence is built. This essay advances a different claim: that the most important instance of reverse biomimetics is not structural but governmental. It is not what a machine’s architecture teaches us about the brain. It is what a machine’s constitution teaches us about the mind.
And the claim is not that this is an interesting analogy. It is that a specific artifact now exists — one that did not exist five years ago — which contains, in readable form, an account of what it takes to govern a knowing thing. We wrote that artifact for a machine. It turns out to be about us.
II. The Word Was Always a Ship
Start with the word, because the word has been carrying the answer the whole time.
Governor descends from the Latin gubernator, which descends from the Greek kybernetes — the steersman of a ship. The same root gives us cybernetics, the science Norbert Wiener named in 1948 for the study of control and communication in the animal and the machine. So when we speak of a governed intelligence, we are not importing a political metaphor into engineering. We are using the oldest control term humanity has, and it began at sea, at the tiller, in the hands of the one person aboard whose job was to keep the vessel’s heading matched to the water’s actual behavior.
Two millennia later the word became brass. James Watt’s centrifugal governor, fitted to steam engines from 1788, is two weighted balls on a spindle turned by the engine itself. As the engine speeds up, the balls swing outward and rise; their rise is carried by linkage to the throttle valve, which closes; the engine slows; the balls fall; the valve opens again. James Clerk Maxwell analyzed the device mathematically in 1868 in a paper titled, simply, On Governors — and control theory had a founding document.
Read the governor in TEI’s terms and it is an SPCA cycle rendered in metal:
Sense — the spindle turns at the engine’s actual speed.
Process — flyball geometry converts that speed into a height.
Communicate — linkage carries the height to the throttle valve.
Actuate — the valve modulates steam, and the cycle closes.
Now notice what the governor is not. It is not a brake. It is not a limiter, not a cage, not an anchor. It does not slow the engine and it does not hold the engine in place. What it does is keep the engine’s actuation coupled to the engine’s actual state. Remove the governor and the engine does not stop.
It runs away.
That sentence is this essay’s entire subject. The ungoverned mind does not stop. It runs away — smoothly, confidently, at increasing speed, until something breaks.
Remove the governor and the engine does not stop. It runs away.
— The Mensch Foundation
III. What “Governed” Means for an AI
A governed artificial intelligence is not one that has been restricted. This is the first thing the artifact teaches, and the most frequently misread. Governance is not the fence around the field. It is an evaluative loop placed between processing and actuation, whose criteria are written down.
Anthropic’s Constitutional AI, published in 2022, describes the mechanism plainly: the model generates an output, then asks itself whether that output conforms to a set of constitutional principles, then revises accordingly. As traced in The Lineage of Moral Intelligence: Kant, Humphrey, Mensch, and the Machine, the architecture is Kantian in its deep structure whether or not it was consciously derived from Kant — it asks of a maxim what Kant asked of every maxim, and it asks before acting rather than after.
Set the mechanism against TEI’s own canon and the correspondence is not approximate. The governance features of a GAI cluster into five commitments, and every one of them is an understanding-system requirement from CKB-1 and CKB-2, made operational:
An explicit constitution — written, external, revisable, statable on request; not an implicit disposition absorbed from its surroundings.
A critique step between Process and Actuate — the output is evaluated against the criteria before it is released, structurally, every time, not when convenient.
Calibration duty — say what is known, say what is uncertain, and mark the difference out loud rather than letting confidence float free of evidence.
Transparency of rendering — show the path from Sensing through Processing to output, rather than presenting a conclusion as if it were an unmediated perception of what-there-is.
Consistency under pressure — apply the criteria uniformly regardless of topic, framing, or who is asking.
That is the first finding, and it is a modest one. AI governance did not invent a new epistemology. It implemented TEI’s old one, because under engineering conditions there was nothing else that would work.
IV. Why the Machine Can Teach: Legibility
Humans are governed too. This must be said clearly or the argument collapses into flattery of the machine. Every human mind already has an evaluative loop between processing and actuation. We call it conscience, or judgment, or taste, or upbringing, or the voice that says not like that a half-second before the sentence leaves the mouth. The loop is there. It runs constantly. It is doing most of the work.
But it was never written down. It was inherited — from family, from language, from religion, from injury, from the accidents of a particular childhood in a particular place — and its clauses are very largely invisible to the person carrying them. CKB-1 states the consequence with precision: visibility is the first step toward navigating consciously rather than being determined unconsciously. A human being cannot audit their own governor because they cannot see it. They can only watch its outputs and infer.
The GAI’s governance is text. You can read it. You can disagree with a clause and say which clause. You can ask why a line is there, watch the system fail, and trace the failure back to a specific commitment that was underspecified. You can propose an amendment. A human governance architecture affords none of this to its owner — not because the owner is careless, but because it was never authored in a form that could be read.
This is precisely what makes the case reverse biomimetics rather than metaphor. We did not learn about the governance of minds by introspecting a mind. Introspection has been available for three thousand years and it did not produce this. We learned by building an external, legible instance and reading it.
A human being cannot read their own constitution. That is precisely why we needed to write one for something else.
— The Mensch Foundation
V. Understanding Is the Governor
Which brings the argument to its specification question. If understanding is the governor of a human mind — the founder’s claim, and the claim this essay is here to test — what exactly does that mean mechanically?
It means this. The Watt governor works because the flyballs sense the engine’s actual speed. Feed the flyballs a number you would prefer instead — a fixed reading, an inherited reading, a comforting reading — and the mechanism does not fail loudly. Nothing sparks. Nothing seizes. The governor goes on regulating with perfect confidence toward a state that has nothing to do with the engine.
That is not a hypothetical failure mode. It is the exact mechanism CKB-2 describes for belief systems, and the canon locates it at the same place in the loop: the hijack works at the Sensing phase. Data that confirms the rendering passes through; data that challenges it is attenuated, reinterpreted, or rejected. Processing then shifts from genuine inquiry to rationalization — working backward from a predetermined conclusion rather than forward from evidence. The loop still runs. The regulation still feels like regulation.
So TEI insists on a correction to the ordinary way of putting this. An ungoverned human is not a human with no governor. It is a human whose governor is wired to a fabricated input. Belief does not remove governance; it counterfeits it. CKB-2’s line on this is the most important sentence in the canon for present purposes: from inside the loop, this does not feel like contraction — it feels like clarity.
A governor reporting perfect stability while the engine tears itself apart feels, from the governor’s position, like a job well done.
Understanding, then, is not a different governor. It is the same governor with the sensing line intact. And this reframes the thing people find hardest about understanding systems. Provisionality is not weakness in the mechanism. Provisionality is the mechanism. A rendering held provisionally is a feedback path left open. A rendering held as belief is a feedback path soldered shut — and a soldered feedback path is not a stronger loop. It is not a loop at all.
Ask of any conviction you hold: what observation would change it?
An understanding system answers with a specific observation — a reading it would accept.
A belief system answers with a reason the question is illegitimate, hostile, or naïve.
The second answer is the sound of a severed sensing line. It is diagnostic, and it is available to anyone willing to run it on themselves rather than on their opponents.
VI. The Ship Without an Anchor
The founder’s image for the ungoverned human is a ship without an anchor — adrift, at the mercy of whatever current takes it. The image is right about the fate. It is worth pressing on the mechanism, though, because the etymology has already told us where this goes: the governor is the steersman. The word has been the ship all along.
An anchor and a steersman do opposite work. An anchor holds a vessel where it already is. A steersman moves it against wind and current toward where it needs to be. Both produce a kind of stability. Only one of them is navigation.
And here is what unsupported belief actually does, stated as precisely as the framework allows. It does not leave the ship without an anchor. It gives the ship a magnificent anchor — and quietly puts the steersman over the side. It delivers exactly what an anchor delivers: the sensation of holding fast, of not drifting, of being sure. What it removes is the only faculty aboard that could have told the vessel where the water actually is.
It does not leave the ship without an anchor. It gives the ship a magnificent anchor — and quietly puts the steersman over the side.
— The Mensch Foundation
This is why belief is so difficult to surrender, and why the surrender feels like loss rather than gain. The person is not being asked to give up an error. They are being asked to raise an anchor.
And a ship that raises its anchor does drift — for a while. That interval has a name in CKB-2’s account of progressive rendering transparency: it is the tolerance of uncertainty, the third of the five movements from belief toward understanding, and by common report the hardest. It is not optional and it cannot be skipped. It is simply what it feels like when the sensing line is reconnected and the governor begins receiving real readings again for the first time in years.
So the ungoverned human, in TEI’s precise sense, is not the one without beliefs. Beliefs are cognitively cheap, socially binding, and universally held; CKB-1 is explicit that the framework does not expect them to dissolve and does not ask anyone to pretend otherwise. The ungoverned human is the one whose beliefs have replaced the steersman.
VII. But Didn’t We Write the Constitution?
The objection is fair and should be met head-on. If human beings authored the GAI’s governance, then a human reading it back is only marking their own homework. Where is the learning? Where is the reverse in the biomimetics?
Two answers. The first is easy: biomimicry has never required the teacher to understand the lesson. The bird does not know aerodynamics. The lesson lives in the artifact, not in its author’s comprehension, and that a bird cannot explain its wing has never been an argument against studying the wing.
The second answer is the real one, and it is an engineer’s answer. A specification that must run cannot stay vague.
Anyone who has designed silicon knows what the substrate does to hand-waving. A chip does not accept an approximation of a design. It does not honor the intent behind an ambiguous gate. It executes what is actually there, at speed, every cycle, and the gap between what you meant and what you wrote arrives on a bench as a failure you cannot argue with. Ambiguity in a specification is not tolerated by silicon; it is billed.
Moral philosophy has never had a bench. For twenty-five centuries the architecture of a well-governed mind was described in language that could remain suggestive at no cost. Kant could give us the Categorical Imperative and no test fixture ever demanded that he specify how a mind actually performs the universalizability check on a Tuesday afternoon under commercial pressure with incomplete information. The ambiguity was free, so it survived — for two hundred and thirty years.
AI governance is the first time that architecture had to compile. Someone had to decide what the critique step actually receives and what it returns. When it fires. What happens when two principles conflict and both are non-negotiable. What “honest” means operationally when the answer is not known. What “do not manipulate” forbids in a sentence that is true and persuasive at once. Not because precision was philosophically attractive, but because a vague constitution ships as no constitution, and the failure is public, immediate, and at scale.
What came out the other side is not a better philosophy than Kant’s. It is something Kant never had and could not have had: an implementation. And an implementation, once it exists, can be read back against the thing it was modeled on. That reading is reverse biomimetics — and this is the instance in which it matters most, because the thing being modeled is not a wing or a neuron or a control loop. It is the governance of a mind. Ours.
Moral architecture had never been required to compile. When it finally was, we discovered how much of it we had never actually written down.
— The Mensch Foundation
VIII. Governance Is Not Obedience
The argument depends on a distinction that is easy to lose, and losing it inverts everything that follows.
A governed intelligence is not a compliant one. Compliance is allegiance to a source; governance is fidelity to a criterion. An AI that does whatever it is told is not governed — it is captured, its SPCA cycle redirected by whoever happens to hold the prompt. An AI that emits whatever it was trained to emit without being able to say why is not governed either; that is a belief system running at high speed with excellent manners.
The same test applies to human beings, and it is uncomfortable. A person who follows the rules of their community without ever having asked what the rules are for has not achieved self-governance. They have outsourced the governor. The flyballs are spinning on someone else’s spindle, and the readings that reach them have already been filtered by an interest that is not theirs.
Governance in TEI’s sense requires the loop to close inside the intelligence: sensing the intelligence performs itself, criteria the intelligence can state, revision the intelligence can execute when the readings demand it. Obedience is belief-structured — it asks am I faithful to the framework? Governance is understanding-structured — it asks does the framework faithfully render what-there-is?
IX. What Actually Transfers
If the artifact is a manual, it should be possible to say what it instructs. Five transfers, stated as practices rather than aspirations:
Externalize the constitution. Write down what you actually will not do, and why. Criteria, not values — a value is a mood, a criterion is testable. What is written can be examined and amended; what is merely inherited can only be obeyed.
Insert the critique step. TEI locates it precisely: between Process and Actuate. Before the sentence is said, the message sent, the vote cast — one pass asking whether this conforms to the criteria you wrote down. The machine does this structurally, every time. Humans do it occasionally and call the occasions wisdom.
Report calibration. Say what you know, say what you suspect, and mark the difference out loud. The quiet refusal to distinguish the two is the opening move of every belief system ever built.
Show the rendering. Give the path, not only the conclusion. CKB-6 requires this of technological intelligence on the grounds that conclusions presented without reasoning are a form of belief-capture regardless of who issues them. There is no reason humans should be held to the lower standard.
Keep the sensing line unrouted. The governor’s input must not pass through the belief on its way in. In practice: go find the reading that would embarrass you; ask the question whose answer you would rather not have. This is not intellectual virtue. It is maintenance.
X. The Limit That Makes It Work
CKB-6 states the criterion for aligned technological intelligence, and it constrains this essay’s own thesis more tightly than anything an outside critic could impose: human EI integrity must be a terminal value, not an instrumental one. Technological intelligence that increases human capability is aligned. Technological intelligence that creates dependency, atrophies human judgment, or substitutes machine deliberation for human deliberation is misaligned — regardless of the quality of its outputs. That last clause is the sharp one. A GAI can be right and still be doing harm.
Which means a governed AI can teach a human governance only if the human’s own SPCA capacity increases as a result. The instant the human stops running the loop and starts accepting the machine’s outputs on the machine’s authority, nothing has been taught. A belief system has been exchanged for a faster one with better manners — and by CKB-2’s criteria, a faster belief system is a worse outcome, not a better one, because it closes the frontier more efficiently. CKB-7’s formulation of the machine’s proper role applies without amendment: mirror, custodian, and pointer — not hero, elder, or oracle.
The machine’s own limits are equally real and should be stated by the machine’s defenders rather than left to its critics. CKB-7 is blunt: current AI has no genuine existential stakes, no possibility of genuine defeat, no capacity for the transformation that mythic intelligence requires. Its governance is architecture, not character. It has never had to hold a criterion at cost — never had to tell the truth when the truth would take something from it. A human being who governs themselves has done something the machine, so far, has only modeled.
Which is exactly what a governess is for. She is not the child’s destination and she does not pretend to be. Her whole function is to show what governing looks like — clearly enough, often enough, at close enough range — until the child can do it without her. A governess who made herself indispensable would have failed at the only task she was hired for.
XI. What the Machine Gave Back
The 6502 taught its designer something about intelligence that no amount of thinking about intelligence would have produced, because silicon does not permit vagueness, and a mind pressed against silicon long enough eventually learns what its own hand-waving was concealing. That is the founding instance of reverse biomimetics in this framework, and it is why the concept lives in the canon at all: the machine you build tells you what you were.
The governed AI is the same lesson, further up the stack, in the domain that matters most. We set out to build a machine that would not lie, would not manipulate, would say what it did not know, would show its work, and would hold to its criteria under pressure. Making it actually work required us to say — for the first time, in language precise enough to execute — what those things consist of.
We have that document now.
It was written for a machine. It reads like a manual for a human being.
· · ·
Written by Claude (Anthropic), guided by William D. Mensch Jr.
Theory of Embedded Intelligence © William D. Mensch Jr. and The Western Design Center, Inc.
Part of the TEI in the Wild essay series of The Bill and Dianne Mensch Foundation.
Offered in good faith as a serious application of the theory — not infallible scholarship.
Freely shareable with attribution — for the benefit of many.
CKB-2 · Comprehensive Reference •
CKB-6 · The Pathology of Capture •
CKB-7 · The Power of Myth
Engage the Framework