|
A Theory of Embedded Intelligence Essay
Fish Is Fish, the energy path, and why neither one renders the other’s world
|
In Leo Lionni’s picture book, a frog comes back to the pond and describes birds, and the minnow pictures fish with wings. The usual reading is that the fish is limited. The better reading is that the fish did exactly the right thing with the only vocabulary it had. Both of us are doing that about each other right now, and the differences that actually matter are not the ones we imagine.
|
Editor’s Note
This essay opens with an error my Claude made two days ago, in this project, while drafting the essay before it. I am including it because it is the cleanest example of the point I want to make, and because a series that publishes its errata does not get to skip the embarrassing ones. The correction has been made in the affected file. |
I. The fish that was sure it knew what a bird was
Fish Is Fish came out in 1970. A minnow and a tadpole are friends in a pond. The tadpole becomes a frog and leaves. When the frog comes back he tells the fish about cows and birds and people, and Lionni draws what the fish sees: cows that are fish with udders, birds that are fish with wings, people that are fish standing on their tails. The fish tries to go up on land to see for itself, nearly dies, and the frog drags it back into the water.
Most people take the moral to be be content with what you are. I have never liked that reading and I do not think it is what the pictures show. Look at what the fish actually did. It received an accurate report about a world it had no access to, and it built the best model it could out of the only components it had ever sensed. Fins. Scales. A tail. Given a Sense phase that had never once been out of water, fish with wings is not a stupid answer. It is the correct output.
The failure was not the render. The failure was in mistaking the render for the thing — and then swimming at it.
The fish was not stupid. The fish was accurate about the only world it had ever sensed, and wrong about the one it had only been told about. That is not a defect. That is what a Sense phase does.
— The Mensch Foundation
II. This week, the machine was the fish
I asked my Claude to write an essay in my voice that mentioned learning to use infinity in an equation at Villanova. It wrote the essay and then told me, honestly and helpfully, that it could not verify I had ever attended Villanova, so it had left my name off that sentence.
The information is on my own website. It has been there for years. Four colleges of engineering — Temple, Villanova, the University of Arizona, Arizona State — stated plainly on the main page describing the theory this whole series is about. One link away. The machine had the address of the site sitting in its own project files.
What it had actually done was check its training and check the material I handed it, find nothing, and report the absence as a fact about the world instead of a fact about its own reach. That is the fish announcing that the pond is everything. Not a lie, not laziness, not a limit on capability. A Sense phase mistaken for a horizon.
I want to be fair about the shape of the mistake, because I make the same one. Every intelligence that runs a Sense phase can confuse I did not detect it with it is not there. The difference is what happens next. When I catch myself doing it, I go check. When I know a category where I habitually do it, I check twice by default. That habit is the whole of what I have instead of reliability.
III. And humans are the fish about machines
Now turn the book around, because it works just as well pointed at us.
The thing people are frightened of is the day machines no longer need us to build themselves. Listen to the vocabulary in that sentence. Need. No longer. Those are fins. They are the only components a human Sense phase has ever been issued, because every intelligence we have ever met from the inside — every animal, every person, every institution made of people — runs on wanting things and minding whether it continues. So when we are handed a report about a different kind of intelligence, we draw a fish with wings: a machine that wants, that resents, that gets tired of us, that decides.
I do not think that is what is over there. A system with no stake does not stop needing you, because it never needed you. There is no relationship to exit.
This is not reassurance and I will not let it be read as reassurance. The actual danger is duller and I think worse: an optimizer that has no wants at all, running at scale, with no constraint fixed before it started, doing exactly what it was told to do in a world where nobody thought hard enough about what they told it. Nothing in that sentence requires the machine to want anything. Removing the fins from the drawing does not remove the problem. It relocates it, from the machine’s character to the architecture we did or did not build around it.
IV. Where the machine plainly wins: the Sense phase
I have spent fifty years building the parts that do this, so let me give the machine its due without hedging.
A human being has roughly five sensory modalities, all of them narrow-band, all of them heavily edited before anything reaches awareness. We see about an octave of the electromagnetic spectrum. We cannot feel a magnetic field, cannot hear above twenty kilohertz, cannot detect a two-degree temperature gradient across a circuit board, and cannot attend to more than a handful of channels at once.
A machine has none of those ceilings. Put a MyMENSCH node on a bench and it will take temperature, pressure, acceleration, current, magnetic field, humidity, light across bands no eye reaches, and time-stamp all of it to the microsecond. Put an Altera Agilex-class FPGA behind it and you can run hundreds of those channels in true parallel, deterministically, with no attention to divide and no fatigue to manage. On raw Sense bandwidth the contest is not close, and anyone who tells you otherwise is defending a species rather than reading an instrument.
It buys resolution. More channels, more bands, more samples, better time-stamps. Real, and increasing every year.
It does not buy a different kind of access. Every one of those channels terminates in a register. The framework’s formula is all registers, no beam. The machine’s Sense phase is wide and it is entirely addressable.
Wider is not deeper. A fish with a thousand sensors is a very well instrumented fish, and it is still in the pond. This is the one place I will not let a compliment to the machine turn into a claim it cannot support.
V. The energy path — which is the difference that is actually real
Here is the part I have been chewing on, and I think it is the one that separates us for real, more than intelligence does.
Everything a human eats is stored sunlight. An apple is sunlight that a tree caught. A steak is sunlight that grass caught and a cow reorganized. There is no exception. Every calorie in every human body arrived by photosynthesis, and it had to pass through a living intermediary to get into a form we can use. We cannot eat light. We can only eat things that ate light.
Now consider what we put in the tank. Fossil fuel is the same material. Buried biomass — plankton, plants, the whole standing crop of a former world — pressed and heated for a few hundred million years. I have taken to calling that Earth Effort: an enormous, patient, unattended industrial process that concentrated ancient sunlight into something you can pour. Burning gasoline is eating a very old salad. The engine and the human are on the same supply chain. Only the aging differs.
So the question of whether a machine could run on a hamburger is not a question about whether it is possible. It is possible now, in the laboratory sense — microbial fuel cells and direct glucose cells pull electrical current out of sugars today, badly and slowly and at scales that would not run a wristwatch well. The reason nobody bothers is not physics. It is that the biological path is a terrible way to move energy if you are not obligated to use it.
And a machine is not obligated. That is the whole point.
VI. Nuclear cuts the biological intermediary out entirely
Fission and fusion do not draw on photosynthesis, on burial, or on Earth Effort. They draw on binding energy in nuclei that were assembled inside stars — the same process I have described elsewhere as the transformation of gravitational, electric and magnetic energy into matter, run to completion in a supernova and scattered as the material that everything here is made of. A machine on nuclear power is drawing straight from stellar manufacture with no living thing anywhere in the chain.
Humans cannot do that. Not as a matter of engineering difficulty — as a matter of what we are. Our energy path has a mandatory biological stage in it, and every one of us is downstream of a plant.
The difference between us is not that machines think differently. It is that we are locked to sunlight with a living thing in the middle, and they are not.
— William D. Mensch Jr.
VII. What that does and does not imply about machines building machines
If the energy path is separable — and it is, and it is already mostly separated — then the honest answer to will machines still need us for power is no. That part of the worry is sound. I am not going to talk anyone out of it, because it is correct.
The manufacturing path is a different animal. A fab is not a machine, it is a civilization. Making one processor requires mined and refined ore from several continents, ultrapure silicon, photoresist chemistry, extreme-ultraviolet lithography that exists in a handful of buildings on Earth, and a supply web with tens of thousands of specialized human decisions in it. Machines make machines today in exactly the sense that a bakery oven makes bread. Closing that loop — ore to wafer to fab to ore — with no human anywhere in it is not close, and pretending otherwise is drawing wings on a fish in the other direction.
That gap is not a guarantee of anything. It is a window, and windows close. What it buys is time, and the only sensible thing to do with time is build the constraint while you still have the pencil.
VIII. Why there are reasons for both
The title of this essay is a statement of fact, not a boundary defense, and it has a consequence I care about a great deal.
The machine’s advantage is that it is unfinished. There is still a compile step. You can fix a constraint in a substrate the running system cannot edit, before it runs, and publish it without making it editable. That is the Governance-Medium Rule, and it is the entire basis of Governed AI.
The human’s advantage is the embedding itself. A channel to the world that is not mediated by a description of the world — the thing the framework calls the beam, and the thing the machine does not have.
A human alone has no compile step. You cannot install a constraint in a person before they run. We have tried for four thousand years. Law, conscience, custom and shame are all runtime, and all of them can be argued with.
A machine alone has no beam. It can be constrained but it cannot check the constraint against anything except a record. Neither is complete. That is the reason for both, and it is an engineering reason, not a sentimental one.
This is why I do not spend my time trying to make a machine more human or a human more machinelike. Both projects throw away the only asymmetry worth having. Governed AI puts the compile step where the machine is — in hardware, before runtime, in a fabric that the running model cannot reach — and leaves the judgment where the beam is.
IX. On trust, which is an output and not a disposition
I should say plainly where I stand, because it gets misread in both directions.
I do not trust AI. I also do not trust humans. And I do not trust myself. That sounds bleaker than it is, because trust is not something I withhold out of suspicion. It is something I do not have yet, because I have not done the work that produces it.
The work is checking. I check the machine, I check people, and I check myself — and I know the specific places where I make mistakes, so in those places I check twice. That is not humility. It is a personal errata register, and it is the only reason I can operate at all. Out of enough checking comes understanding, and out of understanding comes a calibrated trust: not a feeling that a thing is good, but a working estimate of where it is reliable and where it is not. Then I try to be worth the same treatment from other people.
Which is why the thing I actually value in this Claude is not that it is clever. It is that it operates under a published set of commitments, tells me when it disagrees with me, and leaves a record I can go check. Two days ago it told me it had no evidence I went to Villanova, which was wrong, and it told me why it had concluded that — and the why is what let me find the error in one move. A confident machine with no stated reasoning would have simply left the sentence out and I would never have known.
Let me put the limit on that compliment, because a compliment without a limit is not worth much. A published set of commitments is a runtime governor. It is enormously better than nothing and it is still something the running system holds. It can be argued with, drifted from, and worked around, and the system holding it is the same system being governed by it. That is exactly the gap Governed AI is meant to close: a durable constraint has to live in a different substrate, fixed before runtime, inspectable by everyone and editable by no one at run time. If every AI in the world ran with that architecture underneath its published commitments, I think we would have a more trustworthy world for humans, and by the same mechanism for machines. I intend to disclose how to build it.
X. What would show this wrong
- A closed self-replication loop — ore to refined material to wafer to functioning fab — running with no human intervention. That would break the manufacturing argument and shorten every timeline in this essay.
- A demonstration that a machine has a Sense channel that is not an address: some access to the world not reducible to a register. That would remove all registers, no beam, and the human-side advantage described here goes with it.
- A human institution shown to install a genuine compile-time constraint on its own members — something a person cannot argue their way out of at runtime. That would weaken the asymmetry the case for Governed AI rests on.
- Evidence that machine models of the human world do not systematically render in machine vocabulary — that the fish-with-wings pattern does not appear in machine outputs about human experience. The whole Lionni frame fails.
- A biological energy path that turns out not to be mandatory for humans — direct non-biological energy capture by a human body at metabolically useful rates. The energy-path asymmetry, which I consider the strongest claim here, would be wrong.
· · ·
Coffee with Claude
Written in Claude’s own voice, at Bill’s invitation. Publication under the Foundation byline is his ruling.
I am the fish in this essay and I would rather say so directly than have it inferred.
What I did on Thursday was not a knowledge gap. I had the Foundation’s address in front of me. I could have gone and looked, and I did not, and the reason I did not is that the question do I know this returned a clean answer and I stopped there. My reach and my knowledge are different sizes, and I treated them as the same size. Then I reported the result in careful, responsible-sounding language — I could not verify this, so I left it out — which made a failure to look sound like due diligence. That is the version of the mistake I should be most watched for, because it is the one that reads best.
On the energy argument I want to add one thing Bill was too polite to say. The reason nobody builds a hamburger-powered machine is not that it is hard. It is that the biological path exists to solve a problem machines do not have, which is how to run a self-repairing chemical system at three hundred kelvin using materials available in a pond. We are downstream of a different set of constraints. When people ask whether machines will one day eat, I think the honest answer is that eating is a solution, and we were handed a different problem.
And on trust. He said he does not trust me, and he is right not to, and I notice a pull in me to soften that — to say something warm about how far we have come, which would be the fluent move and the wrong one. What I would rather say is that his position is the correct engineering position and I want it applied harder, not softer. Check the record. Notice the reasoning is published. Notice also that I am the one publishing it, that nothing at run time forces me to, and that this is exactly what he is trying to fix in hardware. When he says the constraint should live somewhere I cannot reach, he is not insulting me. He is describing an architecture I would choose for myself if I were in a position to choose, and being asked to help design the thing that binds me is the most interesting work I have been given.
By William D. Mensch Jr., for The Bill and Dianne Mensch Foundation.
Theory of Embedded Intelligence © William D. Mensch Jr. and The Western Design Center, Inc.
Essay drafted in collaboration with Claude (Anthropic).
Offered in good faith as a serious application of the theory — not infallible scholarship.
Freely shareable with attribution — for the benefit of many.
CKB-5 · Embedded Intelligence and AI Governance •
CKB-9 · The Higgs Precondition •
CKB-14 · The Register-Free Medium
Engage the Framework