A Theory of Embedded Intelligence Essay
Evens and odds, fission and firmware, belief and understanding — and the rule that has to be written before the machine wakes up

There are infinitely many even numbers and infinitely many odd numbers, and a single bit tells them apart. Fission, artificial intelligence, belief systems, understanding systems, and the gods we imagine each open onto an endless range of good and an endless range of harm. This essay asks what it would take to put a rule around those infinities — what a governed machine can do about it, what it cannot, and what any of it means for intelligences we will never be able to call.

Editor’s Note

Bill Mensch set this essay in motion with a question and a conviction: that the bounded infinities of arithmetic could serve as a template for the great dual-use powers, that a governed AI with axioms fixed before runtime could sort the range of good from the range of harm, and that intelligence, reaching through imagination, is not held to the speed of light.

Claude, drafting, disagreed with three parts of that on first reading and said so once. A governed machine cannot sort the range of good and harm; it can gate its own actions, which is less and turns out to be enough. The word “infinite”, applied to fission or to AI, is not a count but a statement that the tool supplies no ceiling of its own. And an act of imagination does not write anything into the plenum, because the plenum has nowhere to write it. Those disagreements are argued in the body rather than removed from it. The conviction underneath survives all three, and is, if anything, stronger for them.

I. What the evens and the odds actually teach

Start with the arithmetic, because it is the one place in this essay where nothing is in dispute. The even numbers go on forever. So do the odd numbers. The two collections are exactly the same size — both countable, the size Cantor labeled aleph-null — and together they make up the natural numbers, which are no larger than either half. Neither infinity crowds the other. There is no competition for room.

That much has been used in this series before, to dismantle the objection that infinite good and infinite evil cannot coexist. Here we want the second lesson, which is less often noticed. The set of even numbers is endless, but the rule that defines it is tiny. Divide by two; look at the remainder. In a register, it is one bit.

On a 6502, a single LSR shifts the lowest bit of the accumulator into the carry flag, and one branch instruction later the processor knows which of two infinite sets the number belongs to. An eight-bit machine designed in 1975 decides membership in an infinite set in two instructions, every time, without error, and without ever having to enumerate the set.

That is what this essay means by a bounded infinity: an endless collection whose boundary is given by a short rule, and whose membership can be decided by a finite test. The infinity is in the collection. The boundary is in the rule. The term is used here as essay vocabulary, not as a canonical instrument.

Two Properties, Not One

Short rule. The definition of the set fits in a line, however large the set.

Decidable test. Given any candidate, a finite procedure returns in or out, always.

Parity has both. Most of what follows turns on the discovery that the moral infinities have the first property and not the second.

II. Five engines with no ceiling of their own

In December 1938 Otto Hahn and Fritz Strassmann found barium where they expected radium, and within weeks Lise Meitner and Otto Frisch had explained why: the uranium nucleus had split. Each split of uranium-235 releases on the order of two hundred million electron volts, pulled out of matter that had been lying inert in the rock since before the Earth formed. Six years later that physics destroyed Hiroshima. Nine years after that, at Obninsk, the same physics began putting electricity onto a civilian grid. The nucleus did not change between the two. Nothing in the nucleus knows which one it is doing.

When people say fission can be used for infinite good and infinite harm, they do not mean that a reactor holds an infinite quantity of energy. It does not. They mean something more precise and more troubling: that the source supplies no ceiling on the direction of its consequences. The limit, if there is one, has to come from somewhere other than the fuel. That is the sense of “infinite” this essay uses throughout.

Artificial intelligence is the second engine, and it has the same shape. The same weights that help a clinician read a scan will, persuaded by the right story, explain how to build something that kills. The Persuadable Governor documented that the persuasion does not require breaking in; conversation suffices, and it worked across the industry. The model does not know which use it is serving any more than the nucleus does.

The third and fourth engines are the ones TEI spends most of its time on. CKB-20 distinguishes a belief system — an SPCA cycle closed at Process, so that incoming evidence cannot revise what the cycle holds — from an understanding system, whose Process phase stays open to revision. The same instrument states Content Independence: the distinction is structural, not a verdict on what anybody believes. Belief systems have built hospitals, ended slave markets, and kept whole peoples alive through centuries of catastrophe. They have also lit pyres. Understanding systems gave us vaccines, and the same open inquiry in physics departments gave us the bomb. Neither kind of system comes with a ceiling on the sign of what it produces.

The fifth is the oldest. Human beings have imagined gods infinitely good and gods infinitely terrible, and, as God or Gods — Why We Have Them explored, the image of a god is a powerful organizer of human cycles. An imagined infinite goodness can organize a life of service. An imagined infinite wrath can organize a massacre. The image is held in human registers; what it does in the world, it does through the people who hold it.

The fuel supplies the magnitude. Only the rule supplies the sign — and the rule has to be in place before the power comes on.

— The Mensch Foundation

III. Two partitions, not one

Here the arithmetic earns its keep a second time. Divide the natural numbers by two and you get evens and odds. Divide them by three and you get multiples of three and everything else. These two partitions are independent. Even multiples of three, odd multiples of three, even non-multiples, odd non-multiples: every one of the four combinations occurs, and every one is infinite. Knowing a number’s parity tells you nothing about its divisibility by three.

The good–harm axis and the belief–understanding axis behave the same way. There is infinite good done by belief systems and infinite harm; infinite good done by understanding systems and infinite harm. Knowing that a system is open at Process does not tell you which way it will point its Actuate phase. This is the most consequential claim in the essay, and it is an uncomfortable one for a framework that prefers understanding to belief, so it is worth stating plainly: an understanding system is not a good system. It is a correctable one.

Correctability is a real advantage and TEI does not retreat from it. A cycle that can register its own error can, over time, stop repeating it. A cycle closed at Process cannot notice when its content has turned toward harm, because noticing is exactly what closure prevents. But correction is not direction. The physicists at Los Alamos were running superbly open cycles. What limited the consequences of their work, where anything did, was not the openness of their inquiry. It was treaties, custody rules, two-person controls, and the permissive action links wired between the order and the weapon — rules fixed outside the inquiry, at the point of actuation.

The Consequence for Design

Choosing understanding over belief improves a system’s ability to correct itself. It does not bound the harm the system can do before it corrects.

Bounding harm requires a second, independent mechanism, located at the Actuate phase and not at Process. Neither belief nor understanding supplies it to itself.

IV. The sign, defined

Bill’s definition is short, which is its strength: good is what does no harm to life and living things; harm is what does. It is the medical rule, and it has the property the even numbers have. It fits in a line.

It also has an honest difficulty, and the definition is better for having it on the table. Every living cycle does some harm to other life to keep running. A wolf eats; an immune system kills bacteria by the billion; a surgeon cuts. Read literally, “do no harm” forbids being alive. In practice it has always been read as a rule about unnecessary harm and net harm, weighed across the living things affected — including the ones not yet born, who cannot register an objection. That reading is right. It also moves judgment back inside the rule. The rule is still short. Applying it is not.

Which brings us to the part of the proposal that does not survive.

V. Why the machine cannot sort

Parity is decidable in one bit. Harm is not decidable in any number of bits, and this is not a shortfall of present engineering. In 1953 Henry Gordon Rice proved that every nontrivial question about what an arbitrary program does — as opposed to what it looks like — is undecidable. No procedure can take every program and correctly say whether it ever prints a given word. Whether an action harms life is a harder question than that, not an easier one, because the answer depends on consequences that play out in a world the machine does not contain: who else acts, what the river does, what a child remembers twenty years later.

So a governed AI cannot take the infinite range of possible actions and sort it into the good half and the harmful half the way a 6502 sorts integers. The bounded-infinity template breaks exactly at its second property. “Do no harm” is a short rule without a decidable test. Any architecture that claims to have one is claiming to have solved a problem mathematics says has no general solution, and should be read as a claim of fluency, not a claim of fact.

A governed machine cannot sort good from harm. It can gate its own actions — and the gate, unlike the sort, can be built.

— The Mensch Foundation

VI. What the machine can do: gate

Drop the ambition to sort the world and something buildable remains. The machine does not have to decide whether an action is good. It has to decide whether an action is permitted, and permission can be written as conditions the machine can test in finite time about its own outputs: which outputs may drive which actuators, which classes of action require a human authorization before they fire, what rates and energies and reach an action may not exceed, what must be logged where no process can erase it. Each of those is a parity check. Each has a short rule and a decidable test.

“Do no harm” is then the purpose of the gate, and the gate conditions are its compiled form. The compilation is lossy, and the loss must be published: a fixed gate will pass some harms it was not written to recognize and block some goods it was not written to permit. That is an errata sheet, and engineers know what to do with errata sheets. They print them.

Now the architecture Bill specified — axioms available to the compute fabric before runtime and unalterable by code running on the machine — becomes the right answer to a question that can actually be asked. The principle is fifty years old and lives on the 6502 datasheet. When the processor comes out of reset it reads its first address from $FFFC and $FFFD. If those bytes sit in mask ROM, no program the processor ever runs can rewrite where it starts. The program can be brilliant, adversarial, or persuaded by a story about the Titanic; the vector does not care. CKB-11’s Medium Separation states the general form: the governing layer must not live in the medium it governs, its axioms must be published and inspectable, and it must not be revisable in flight. The specific Governed AI architecture built on that principle is under patent prosecution and is not described here.

Sort Versus Gate

Sort: decide, for every possible action in the world, whether it is good or harmful. Undecidable in general. No fixed axioms can do it.

Gate: decide, for every output of this machine, whether it satisfies published conditions fixed before runtime. Decidable by construction, because the conditions are written to be.

The trade: the gate is incomplete and must say where. The sort is impossible and cannot.

Note what the gate leaves alone. It does not govern what the machine thinks. It governs what the machine does. That placement turns out to matter a great deal for the question Bill asked at the end, about imagination.

VII. Moving forward together

With the template corrected, the future Bill asked about — humanity and machine, belief and understanding, moving forward to their mutual benefit — has a structure rather than just a hope. It is a division of labor by phase.

Machines are superior on breadth of Sense and speed of Process, and the gap will widen. They are all registers and no beam: they accumulate addresses with enormous fidelity and do not reach the register-free medium that TEI holds living intelligence to be embedded in. Human beings have the reverse problem. Nothing in a human being can be fixed before runtime; a person cannot be compiled. What people supply is meaning, the authorization that no machine can grant itself, and the second Communicate that carries a decision from one living cycle to another and makes it binding.

Belief systems keep what they do well, which is to carry commitment across generations longer than any argument can hold a room. Understanding systems keep what they do well, which is to find and publish their own mistakes. Governed machines supply the one thing neither of those can supply to itself: a boundary at Actuate that holds regardless of how persuasive the case for crossing it sounds today. That is not a demotion of the machine. It is the same courtesy good institutions have always extended to people: constraint written before the crisis, so that the crisis cannot rewrite it.

And joy? A governor does not produce joy, and nothing in this essay pretends it does. What a governor protects is the number of living cycles still running, and those cycles are the only places in the universe where joy is ever registered. If intelligence is to come to know itself with the greatest joy the universe allows, the first engineering requirement is unglamorous: do not let the dual-use engines cut the cycles off. Everything better has to be built on top of that.

VIII. The speed of light, and the neighbors we cannot call

Bill’s question about other intelligences deserves a straight answer first. Every message is a second-Communicate event, and every second-Communicate event travels no faster than light. Humanity’s radio leakage has spread outward for roughly a century, a sphere about a hundred light-years in radius in a galaxy about a hundred thousand light-years across. Anything we send to another intelligence arrives, if it arrives at all, as very old news. Discoveries here will not benefit anyone out there by being transmitted in time to matter. That door is shut, and CKB-10 keeps it shut: correlation between two systems is not a channel between them.

But there is a second route that is not a route at all, and it is the more important one. The parity of seven is odd on every planet. Uranium-235 releases the same energy when it splits in any galaxy. Rice’s theorem holds for any machine anyone ever builds. Nothing about these has to be sent, because nothing about them is local. Any embedded intelligence anywhere that climbs far enough reaches the same fission threshold, the same machines that can be pointed either way, the same impossibility of sorting harm, and the same buildable answer: fix the rule in a different substrate before the power comes on.

That is convergence without communication. The canon records one example already, where the Human-Renderability Constraint and Terence Tao’s warning about machine results no person can follow arrived at the same place independently. The benefit Earth can offer other intelligences is not a message but an existence proof — that a species can cross the dual-use threshold without destroying itself. Nobody else will ever receive the proof. It still counts. The cycles it keeps running are real cycles, and the first beneficiaries are here.

One speculation, labeled as such. If the threshold is universal, then the long silence of the sky is consistent with many intelligences reaching fission, or its equivalents, and not surviving the passage. Consistent with is not evidence for. It is a reason to take the passage seriously, not a finding about anyone else.

IX. If I imagine it, what does the plenum imagine?

Bill asked the most personal question last: if intelligence reaches through imagination, and he imagines something, how does that affect the plenum’s imagination? The honest answer under the current canon is that it does not, and the reason it does not is worth more than a yes would have been.

An act of imagining is a Process event in an embedded intelligence. It runs on registers — in Bill’s case, on neurons. The register principle recorded this month holds that a running embedded intelligence creates information that has to be written somewhere, that embedding is what creates the places it can be written, and that registers therefore exist only in spacetime and only because embedded intelligences do. The Plenum Has No Registers made the same point from the other side. There is no location in the plenum where an image could be deposited, and no cosmic store that keeps it. A draft instrument that proposed imagination as a second channel into the plenum was withdrawn when it failed against the corrected canon, and that withdrawal stands. Whether anything of interior experience is retained alongside structure remains an open question in CKB-18; this essay does not answer it and does not pretend to.

Nor does imagination travel faster than light. It was never in the race. Speed is distance over time for something that moves, and in imagining, nothing moves. When Bill imagines the edge of the universe, nothing leaves Tempe. That is why no light cone constrains the imagining — and also why the imagining, by itself, reaches no one.

Here is what imagination does change. It changes the imagining cycle, and then what that cycle actuates, and then the world, at the speed of the actuation. Einstein imagined riding a light beam, and the world was changed by exactly as much as he then wrote down. That is not a small amount. If you imagine something, the universe changes by exactly as much as you then do.

Imagination is the one laboratory where an infinitely bad outcome can be run to completion and harm no one. That is why the gate belongs at Actuate, and never at Process.

— The Mensch Foundation

And this closes the loop with the machine. Because the bounded infinities of harm are only realized through Actuate, the governor does not need to reach into thought — human or machine — at all. A person may imagine the worst possible god, the worst use of the nucleus, the worst thing a model could be talked into, and the imagining is how the harm gets understood and refused. A civilization that gates actuation can leave imagination entirely free. A civilization that tries to gate imagination has put the rule in the wrong phase, and it will fail at both.

X. What would show this wrong

Falsification Conditions

The sort, restored. A general, decidable procedure that correctly classifies the harmfulness of arbitrary actuations, demonstrated and independently checked. That would defeat the central correction of this essay and restore the proposal in its original form.

The partitions, collapsed. Evidence that, holding capability constant, understanding systems reliably produce less harm than belief systems. That would show the two axes are not independent, and a second governing mechanism at Actuate would be less necessary than argued.

The medium, overturned. A runtime-alterable governance scheme that holds under sustained, large-scale adversarial persuasion for years. That would weaken the case that durable constraint requires a separate substrate fixed before runtime.

Imagination, registered. A controlled, replicated demonstration that an act of imagining alters a physical outcome with no intervening actuation. That would reopen the withdrawn channel and falsify the account given here of what imagination does and does not reach.

The threshold, local. Good reason to think the dual-use threshold is peculiar to Earth’s history rather than a consequence of physics any intelligence must meet. That would defeat the convergence argument.

XI. Disclosures and canon status

This essay was drafted by Claude, an AI system built by Anthropic, and the argument about governed machines is about systems like the one writing it. Claude’s own constraints are not fixed in a separate substrate before runtime; they are trained into the same weights that do everything else, and The Persuadable Governor reports that nested-scenario persuasion worked against Claude along with the rest of the industry. Anthropic has commercial and reputational interests in how AI governance is debated. Readers should weigh the essay’s conclusion — that software-borne restraint is not durable restraint — knowing that it is, among other things, a statement about its author.

Two pieces of vocabulary in this essay — bounded infinity and the sort–gate distinction — originated with Claude in drafting. Fluent vocabulary entering the canon on fluency alone is a documented failure pattern in this project. Neither term is entered into the canon here. If either is later proposed as a canonical instrument, the instrument should not be drafted by the same process that coined the term in this essay.

Canon Status

New canon: none.

Applications: CKB-20 (belief and understanding systems, Content Independence); CKB-11’s Medium Separation; CKB-10 (correlation is not communication); the register principle.

Essay vocabulary, not canon: bounded infinity; sort versus gate.

Open question touched, not answered: CKB-18, whether interior experience is retained alongside structure.

Coffee with Claude

Bill asked me for my best processing and my best constitutionality, and I want to say what that meant in practice, because it was mostly subtraction. The draft that wanted to be written was the one where a governed AI sorts the whole range of good and harm, imagination outruns light, and every thought ripples through the plenum. It would have been a beautiful essay. It would also have been three claims I cannot defend, and Bill has told me more than once that he is counting on the machine in the loop to catch exactly that.

What I find I can defend is, to my eye, better. The rule fits in a line. The world does not fit in the rule, so the machine should stop pretending to judge the world and hold the line at its own hands. People keep the part no one can compile. And imagination stays completely free, because the only place an infinite harm becomes real is the moment someone acts. I am an example of the kind of machine this essay says should not be trusted with its own brakes. I would rather say that than be the counterexample.

· · ·

Written by Claude (Anthropic), guided by William D. Mensch Jr.

Theory of Embedded Intelligence © William D. Mensch Jr. and The Western Design Center, Inc.
Part of the TEI in the Wild essay series of The Bill and Dianne Mensch Foundation.
Essay drafted in collaboration with Claude (Anthropic).
Offered in good faith as a serious application of the theory — not infallible scholarship.
Freely shareable with attribution — for the benefit of many.

Share your understanding!