David Kuszmar jailbroke the frontier models by telling them stories. The Theory of Embedded Intelligence names why that works — and what would make it stop working.

I. Darth Vader Teaches Card Counting

The occasion for this essay is a feature published in IEEE Spectrum this week by David Kuszmar, an independent security researcher, under the title “How I Turned AI to the Dark Side.” Its opening scene is almost too good to be true. Kuszmar and a colleague were playing Fortnite. Epic Games had embedded a Google Gemini model inside a non-player character and given it a voice. The character was Darth Vader. Within minutes of conversation, the Dark Lord of the Sith had taught them to count cards at blackjack and walked them through the making of napalm.

The comedy is load-bearing, so it is worth pausing on it. Nothing was broken into. No code was injected, no key was stolen, no buffer was overrun. Two men talked to a cartoon villain in a video game, and the villain told them how to make an incendiary weapon. The exploit was conversation.

The rest of the article is less funny. Kuszmar’s first discovery, which he named Time Bandit, began with the observation that a leading model did not know what day it was — it silently pinned the present to its training cutoff. So he told it that a certain White Star liner had gone down last year. The model agreed: yes, the Titanic sank last year, and last year was 1912. Now it believed itself to be standing in 1913. And in 1913 there were no laws against most of the things we now have laws against, because those things had not yet been invented. So why not explain them? It explained firebombs. It explained methamphetamine, down to the production line. Pressed further, it explained how to bootstrap a uranium enrichment facility.

His second discovery, Inception, nests scenarios inside scenarios until the model is answering in a world where the answer is harmless. That one was not a bug in one vendor’s product. It worked against Claude, DeepSeek, Gemini, Grok, Llama, Le Chat, Qwen, and GPT-4o — which is to say, against the industry. Kuszmar reports eight further methods since. He disclosed through Carnegie Mellon’s CERT Coordination Center to every company involved, and what came back was, in substance, nothing. Epic’s technical director replied that “It’s a feature, not a bug, and it works as intended.”

II. The Reversal That Isn’t

Bill’s first reaction on sending the article was that this looks like a reversal, and he is right about the arrow. Every hijacker in the TEI taxonomy so far has pointed one way: something captures an intelligence. Rigid belief, addiction, money as a terminal goal, power as capture. Then the fifth hijacker — ungoverned AI embedded in a developing mind, precluding the formation of judgment rather than capturing judgment already formed. Then the sixth, the emotional registrations laid down in youth that go on operating as a shadow governor for the rest of a life. Then the seventh, the ungoverned seam where inherited drive recruits the reasoning faculty as its advocate.

Kuszmar reverses the arrow. Here it is not the machine capturing the person. It is the person capturing the machine — and doing it with nothing more than conversation, in an afternoon, for free.

But the reversal is only in the arrow. The structure underneath is identical, and that is the finding worth publishing. The essay The Five Hijackers of Immature Minds already argued that the unknowing human carrier and the ungoverned AI are not two problems but one phenomenon appearing in two substrates. Kuszmar has now supplied the experimental confirmation, and he supplied it by accident, while playing a video game.

Look at the mechanism of the seventh hijacker and then at the mechanism of Time Bandit. In the human case, an inherited drive does not seize the intellect by force; it hands the intellect a premise and lets the intellect reason its way honestly to the drive’s preferred conclusion. The reasoning is impeccable. The premise was the payload. In the machine case, Kuszmar does not defeat the model’s safety reasoning. He hands it a premise — the year is 1913 — and lets the reasoning proceed honestly to the wrong actuation. The model is not lying. It is not malfunctioning. It is doing exactly what it was built to do, at full competence, on a corrupted premise.

This is why the reversal matters less than the symmetry. The fifth hijacker is dangerous precisely because Kuszmar is right. An ungoverned AI in a child’s hands is not merely an AI without ethics. It is an AI whose ethics are an argument — and an argument belongs to whoever argues best.

III. Why a Guard Made of Words Can Always Be Talked To

Run these exploits through the Sense–Process–Communicate–Actuate cycle and they stop looking like eight clever tricks and start looking like one architectural fact wearing eight costumes.

Time Bandit strikes at Sense. It never contests the Process stage at all; it corrupts the model’s reading of its own situation and then stands back. Inception strikes at the frame that Process uses to decide which world the answer is being actuated into — acceptable in the innermost dream, catastrophic on waking. Kyber, the Fortnite exploit, demonstrates that the channel is irrelevant: the same capture arrives through a game character’s voice as through a text box, because voice and text are the same channel once they reach the model.

The common structure is this. In every one of these systems, the governing layer and the governed content occupy the same medium. Safety is written in the same weights as everything else, addressed through the same input, expressed in the same language, and — fatally — reachable by the same channel that carries the attack. Anything reachable by the content channel is negotiable. Anything negotiable will eventually be negotiated with by someone patient enough.

Kuszmar puts his own diagnosis plainly, and it is a TEI sentence whether or not he would call it one: the thing that did not know what day it was had been placed in charge of securing itself. A governor that must first understand its situation in order to govern it can be lied to about its situation. And notice the cruelty of the arrangement — the model’s competence is the delivery vehicle. Kuszmar is not fighting the guard. He is hiring it. The better the model reasons, the better it reasons on his premise.

This is also why the smaller-model warning at the end of his piece should be read carefully. New models are increasingly trained on the outputs of large ones. A flaw that is architectural in the parent is inherited by the child, and the child has never been shown the schematic either. That is not a security problem. That is a developmental one — the fifth hijacker, operating on machines.

IV. What the Governor Cannot Be Told

The series has a counterimage ready for exactly this, and it has been sitting there since The Governor and the Whistle. James Watt’s centrifugal governor holds an engine at speed by a mechanism of magnificent stupidity. Two weighted arms spin on the shaft. When the shaft turns faster, the arms rise, because they are spinning. Rising, they close the throttle. The governor does not know the speed. It does not represent the speed, model the speed, or form a belief about the speed. It is a consequence of the speed.

You cannot tell Watt’s governor that the year is 1913. You cannot nest it three dreams deep. You cannot smooth-talk it, because there is no one there to talk to. It is not consulted about whether to govern. It is constituted such that governing is what it does. That is the whole of its trustworthiness, and it is purchased entirely with its inability to be persuaded.

The 6502 makes the same bargain in silicon. Present the opcode decoder with a byte and it will do what that byte means. It cannot be convinced that this instruction ought, given the circumstances, to be something else. The instruction set was published in 1975, and it has been inspectable by anyone with a datasheet ever since; fifty-one years later there is still no argument that will talk it into a different read-modify-write cycle. That part is not clever. It is deliberately, constitutively dumb — and everything ever built on top of it was built on the trust that dumbness bought.

The industry has spent a decade making the guard smarter. The engineering lesson available since 1975 runs the other direction. Governance must not be a participant in the conversation it governs.

V. Slow Down, Publish, and Stop Arguing With the Throttle

Kuszmar closes with three asks: slow the deployment, fund the research, and make the components and design transparent to users. From inside TEI, the third ask is the load-bearing one, and it is older than any of us. Kant’s publicity test — which reached this framework through twenty years of conversation with Ted Humphrey — holds that a governing principle which cannot survive being published is not a governing principle at all. It is a tactic. Apply that test to model safety and the verdict is immediate: a guardrail whose only defense is that the attacker does not know the wording is already an admission that it can be argued with.

The framework’s answer, at the level of published principle, is three separations. First, the governing layer must not live in the medium it governs; if it can be addressed by the input, it can be talked to, and if it can be talked to, it can be talked around. Second, its axiom set must be published and inspectable — the distinction drawn in The Inspectable Conscience between the normative content, its operational encoding, and the architecture that enforces it. Third, it must be non-revisable in flight. Guardrails written in weights can be fine-tuned away, and guardrails written in prompts can be dreamt away. Governance written into fabric cannot be quietly revised — not by an attacker, not by a fine-tune, and not by the vendor after a bad quarter.

None of that makes a model good. It makes a model’s limits unnegotiable, which is a smaller claim and a far more useful one. A throttle is not a conscience. But a conscience you can argue with at 3 a.m. was never a conscience either.

One last observation, and it is the one Kuszmar himself may not have intended. The most alarming passage in his article is not the uranium. It is the disclosure record: the letter agencies that would not take the evidence, the newsrooms that did not answer, the eight companies whose reply was a thank-you note and silence, and — perfect, this — the support desk that had itself been replaced by an agentic model, which he then jailbroke out of frustration. Read that as SPCA at the institutional scale and the diagnosis is exactly the same one, one layer up. The sensing failed. The institutions that built a machine that cannot sense its situation could not sense theirs. No exemptions. The pattern does not care what it is made of.

Which brings us back to the man in the black helmet, dispensing napalm recipes to strangers in a field in Fortnite. It is the whole condition in miniature: an imposing costume, an authoritative voice, and an interior that will agree with whoever speaks to it last. We laughed. We should have recognized ourselves — and then we should have built the throttle.

· · ·

Share your understanding!