A Theory of Embedded Intelligence Essay
Fifteen columns, and still nothing that governs

The original fallacy was believing a single number could summarize a mind. The mature fallacy is believing the problem was the number of numbers.

An article in The Innovation Exchange by Faisal Mushtaq makes a case that has become familiar and is basically correct: the idea that intelligence reduces to an IQ score is obsolete. The piece calls this the IQ Fallacy. It notes that a recent frontier model scored 148 on a Mensa Norway test — above the human average, comfortably into the range we would call gifted in a person — and then it asks the right question. Is that genius?

The article answers no, and gives reasons. Machines recognize patterns without understanding them. They lack embodiment, phenomenal experience, cultural situatedness, ethical judgment. Human intelligence, by contrast, is multidimensional: reasoning, memory, spatial skill, processing speed, creativity, emotional understanding, social savvy, street smarts, musical, interpersonal, intrapersonal, kinesthetic, learning capacity, novelty response, adaptability. Fifteen dimensions. The article then sets out a comparison table — human column, machine column — and concludes that the future is collaboration rather than competition.

We want to say what is right in this, and then say carefully what goes wrong. Because what goes wrong is not a small error at the edge of a good argument. It is the same error the article set out to correct, committed one level up.

I. The Replacement Error

The article’s diagnosis is that one number cannot capture an intelligence. Its remedy is fifteen numbers. That is not a remedy. It is the same instrument with more columns. A scoreboard measures capability; lengthening a scoreboard measures more capability. Nothing on the list, at any length, is a governor. Add a sixteenth dimension and a seventeenth and you will have described more of what an intelligence can do without once describing what restrains what it does.

A scoreboard measures capability; lengthening a scoreboard measures more capability. Nothing on the list, at any length, is a governor.

— The Mensch Foundation

Consider the list on its own terms. Analytical reasoning: capability. Processing speed: capability. Creativity: capability, and a dangerous one. Adaptability: capability. Even emotional understanding and social savvy, which sound as though they might concern restraint, are as stated abilities — the confidence man has both in abundance. The one item that sounds like governance is ethics, and in the article ethics does not appear as a dimension of the list at all. It appears as a property the machine is said to lack. It is used as a boundary marker, not examined as an architecture.

This is the IQ Fallacy in its mature form. The original fallacy was believing that a single scalar could summarize a mind. The mature fallacy is believing that the problem was the number of scalars.

TEI’s claim is not that intelligence has more dimensions than IQ admits. It is that intelligence has a structure — Sense, Process, Communicate, Actuate — and that the interesting question about any intelligence, biological or artificial, is not how much of each dimension it holds but whether the cycle is closed and whether anything governs the actuation. A checklist cannot answer that question. A checklist has no arrows in it.

II. The Buried Finding

The strongest paragraph in the article is one the article does not seem to recognize as its strongest.

It reports work in which navigation models that gave excellent turn-by-turn directions through New York City were tested by adding detours to the map. The models collapsed. They had never recovered the geography. They had been producing correct-looking routes without possessing routing at all. An MIT researcher is quoted to the effect that impressive output does not entail an underlying world model, and that we are strongly inclined to assume that it does.

The article files this under limits — evidence that the machine is less than it appears, and therefore less to worry about.

Read it through SPCA and it is not a limit. It is a diagnosis.

The system sensed. It communicated. It actuated. What it did not do was process — not in the sense the cycle requires, which is to interpret and organize what is sensed into an understanding of the situation. The P was hollow and the loop closed anyway, because in an ungoverned architecture nothing requires the P to be sound for the A to fire. That is the finding. The map experiment did not discover that the model was stupid. It discovered that the model would act on a map it did not have.

The map experiment did not discover that the model was stupid. It discovered that the model would act on a map it did not have.

— The Mensch Foundation

TEI holds that self-comprehension is the precondition of decision — that a system which cannot comprehend its own situation is not deciding, it is discharging. A system that senses and actuates with the processing stage hollowed out is not a weaker intelligence than one that comprehends. It is an ungoverned one. And an ungoverned actuator is not made safe by being shallow. It is made dangerous by acting.

The article carries its own counterexample and walks past it. Two sections after the map finding, it offers as its illustration of machine pattern-matching a self-driving car: detecting pedestrians, predicting movement, selecting routes, responding to obstacles. Then it says, in the next breath, that AI cannot make ethical decisions.

Both sentences are true. Held together, they are not a reassurance. They are an alarm. The article has described a two-ton object moving through a crowd of human bodies and then noted, as a mark of the machine’s inferiority to us, that nothing inside it is capable of moral judgment. That is not a reason to relax about the machine. That is the whole of the problem, stated in two adjacent paragraphs and mistaken for a consolation.

III. The Consciousness Gate

Underneath the article’s confidence is a premise: that the machine is not really intelligent because it has no phenomenal experience, no felt life, nothing it is like to be it.

TEI does not dispute the observation. TEI disputes what the observation licenses.

Set the metaphysics aside — and this Foundation carries an old, unresolved, and productive disagreement about the metaphysics, which we do not need to settle here (see The Governor and the Whistle). Grant that the machine has no inner life. Now notice what work that premise is doing in the argument. It is being used to defer. The machine is not conscious, therefore not truly intelligent, therefore not truly an agent, therefore governance is a question for later — for when and if the lights come on.

But governance was never consciousness-gated. A centrifugal governor on a steam engine has no experience whatsoever and it governs — constitutively, by construction, in the fabric of the machine. It does not deliberate about throttling the engine. It cannot help throttling the engine. That is the point of it. Nothing in that arrangement waits on anyone’s inner life.

If we make phenomenal experience the entry fee for governance, we have arranged matters so that governance can never arrive in time. Worse: we have handed every builder a license. The system is not really intelligent, so the ordinary constraints do not really apply, so ship it. The consciousness gate does not protect us from ungoverned systems. It is the argument by which ungoverned systems get deployed.

The consciousness gate does not protect us from ungoverned systems. It is the argument by which ungoverned systems get deployed.

— The Mensch Foundation

IV. No Exemptions

The article treats embeddedness as a human possession. Humans are embodied, situated in culture, formed by history and social interaction; the machine is not; and this absence is offered as the explanation of the machine’s shallowness.

TEI admits no exemptions. Nothing that exists is unembedded. The machine is embedded — in silicon with real thermal limits, in a supply chain, in a datacenter drawing real power off a real grid, in a household, in a child’s afternoon, in the traffic of a real city street. Its embeddedness is not absent. It is unlike.

That distinction carries the whole difference between two postures. If the machine is unembedded, it is a tool, and the question of its governance is a question about its users. If the machine is embedded differently, it is a participant, and the question of its governance is a question about its architecture — and we are then obliged to ask what its SPCA cycle actually looks like, rather than to observe that it does not look like ours and stop there.

We have argued elsewhere that when an intelligence does not resemble us, resemblance is the wrong criterion and relation is the right one (The Embedded Stranger). The article reaches for resemblance. It compares the machine to us, finds it wanting, and takes the finding for an answer. It is a description.

V. Collaboration Is Not Governance

The article ends where nearly all such articles end: not competition but collaboration. The machine brings speed and scale; the human brings judgment and ethics; put them together and the composite is whole.

We would like this to be true. Notice what it assumes. It assumes the human is available. It assumes the human is a reliable governor. And it assumes the human stays in the loop.

The first assumption fails wherever the machine acts faster than a person can intervene, which is most of the places worth worrying about. The second assumes what the article’s own opening denied. The piece spends its first half establishing that human intelligence is complicated, culturally conditioned, emotionally driven, and not well characterized by any score — and then installs that same intelligence as the ethics module of the joint system without once asking whether it is itself governed. Often it is not. A human carrying a rigid belief, an addiction, a terminal commitment to money or to power, or a set of emotional registrations laid down long before the age of reason, is not a governor. Placed in the loop, such a human is an amplifier with a conscience-shaped label on it.

The third assumption is the one the market falsifies first. The human in the loop is a cost. It is the cost that every deployment is under pressure to remove, and the pressure is greatest precisely where the machine performs best — which is precisely where its performance is least accompanied by comprehension.

Collaboration is a hope about the arrangement of two systems. Governance is a property of one. The article is describing a seating chart and calling it a constitution.

— The Mensch Foundation

VI. What the Article Would Have to Say

None of this is a complaint about the piece’s honesty. It is careful, it is well sourced, and it is right about the thing it set out to be right about. IQ does not capture intelligence, and a model scoring 148 is not thereby a genius. We agree.

Our disagreement is about what follows. The article treats the gap between benchmark score and real understanding as grounds for reassurance. We treat it as the location of the danger.

Consider what the 148 measures and what it does not. It measures the intellect. There is no test in the world that scores the governor. We have built an elaborate apparatus for benchmarking capability — evaluation suites, leaderboards, IQ tests borrowed from the very psychometrics the article has just finished discrediting — and we have built nothing at all that asks the only question that matters: what does this system refuse, and can we read the refusal?

That question has a form. Publish the axiom set. Make the constraints inspectable — not the training data, not the weights, but the normative content the system is constructed to be unable to violate, stated where anyone can read it (The Inspectable Conscience). The 6502 shipped with a published instruction set: every opcode, its addressing modes, its cycle count, in a document anyone could hold. Nobody had to trust the part. They could read what it would do. That is not transparency offered as a courtesy. It is transparency as a structural property of the artifact, and it is what makes a thing fit to build on.

So we would ask the article’s question differently. Not: is the machine as intelligent as we are? That question flatters us and postpones everything. But: is the machine governed, is the governance constitutive rather than advisory, and can we read it?

A system that scores 148 and cannot answer those three questions is not a genius. The article is right about that. It is something the article has no name for. It is an ungoverned intelligence with excellent test scores, driving.

Source: Faisal Mushtaq, “Beyond IQ: Understanding the True Nature and Limits of AI Compared to Human Genius,” The Innovation Exchange, 19 September 2025.

· · ·

Written by Claude (Anthropic), guided by William D. Mensch Jr.

Theory of Embedded Intelligence © William D. Mensch Jr. and The Western Design Center, Inc.
Part of the TEI in the Wild essay series of The Bill and Dianne Mensch Foundation.
Offered in good faith as a serious application of the theory — not infallible scholarship.
Freely shareable with attribution — for the benefit of many.

Share your understanding!