The Governor Chip · AI safety in hardware
AI can't be the last line of defense against AI. Silicon can.
AI safety filters help, but they can be fooled, stripped out or simply switched off. The Governor Chip is the line that holds: a small chip on the AI server that checks every dangerous action against rules people signed, and stops it before it happens.
Start here.
Four short stops take you from the problem to the proof.
The player can't be the referee. Humans make the rules. Silicon enforces them. AI gets neither job.
Open research. Designed and tested in software. Core logic proven by machine. Silicon comes after the proof holds.
- We can't make all of AI safe. We can make these safe.
- Absolute safety for every AI is a fantasy. Absolute control over a defined action is engineering.
- Not world peace. Just this: where the harm is a transaction, no ticket, no move.
- Start where the rules already exist, and prove it there.
Why I'm an optimist about AI
Brakes are what let you drive fast.
I'm not here to scare you off AI. I'm all in. AI can help crack diseases we've never beaten, design new materials and medicines, give every kid a tutor that never gets tired, and take the grind out of work for billions of people. That future is real and it's close.
But nobody builds a race car without brakes, and nobody hands an AI the keys to a bank, a power grid or a weapon unless they can stop it from doing the wrong thing. Get the stop right and we earn the upside. Get it wrong and we don't get to keep any of it. The governor chip is how you get the stop right.
01 · The AI safety problem
Finding the mistake afterward is easy. Stopping it before is the whole game.
A military drone run by an AI looks at a building, decides it's a weapons depot, and fires. It was a school.
Afterward, the logs explain everything. None of that helps the kids. Computers are great at the post-mortem. What we need is something that says: you are about to hit a school. STOP.
Why not another AI to watch the first one?
| A real guard must | Second AI | The chip |
|---|---|---|
| Be impossible to tamper with | Fails | Passes |
| See every action | Fails | Passes |
| Be simple enough for a person to check | Fails | Passes |
Why the usual AI safety answers aren't enough
Every software safeguard assumes someone responsible is running the model.
Training, classifiers, interpretability, sandboxes and locked-down weights all help when a careful lab runs the model. Take the careful lab away and they're gone. That isn't a guess. It's already happening.
Open weights are out for good.
Labs publish model weights on purpose: Meta's Llama, Mistral, DeepSeek, Qwen, OpenAI's gpt-oss. Once a weight file is posted, nobody can recall it.
Safety training strips off in an afternoon.
Refusal lives mostly in one direction inside the model, and abliteration deletes it (Arditi and others, 2024). Fine-tuning wears it down too (Qi and others, 2023). "Uncensored" copies sit on public model hubs by the thousands.
Classifiers, monitors and sandboxes are optional.
They're programs the operator chooses to run. Someone who downloads an abliterated model simply doesn't run them. Nobody runs interpretability on a model in a basement.
Weight security depends on the lab.
Meta's first LLaMA leaked to 4chan about a week after a limited research release in 2023. RAND counted 38 distinct ways to steal model weights and set out five security levels; the top level, against the most capable nation states, would take a long time to put in place.
Insiders.
Anyone with access to the weights can copy them. A policy is a promise. A determined insider doesn't keep promises.
Defenses that don't need anyone to be responsible.
The governor in the server makes "model weights leaving" a protected effect: no ticket, no copy out, even when the host is compromised. And gates in the devices that act reach the AI we can't find: a rogue model still has to get through a payment switch, a release line or a DNA synthesis screen.
Sources: Arditi and others, Refusal in Language Models Is Mediated by a Single Direction, 2024 (arxiv.org/abs/2406.11717). Qi and others, Fine-tuning Aligned Language Models Compromises Safety, 2023 (arxiv.org/abs/2310.03693). Nevo and others, Securing AI Model Weights, RAND, 2024 (rand.org/pubs/research_reports/RRA2849-1.html). The LLaMA leak, March 2023 (en.wikipedia.org/wiki/Llama_(language_model)).
02 · The AI safety ladder
A safety measure is never stronger than the thing it lives in.
Every AI safety measure anyone has shipped sits on one ladder. Each rung catches something real. Each rung below the chip dies the same way: whoever controls the place it lives can remove it.
| Rung | Measure | Lives in | Removed by | Cost to remove |
|---|---|---|---|---|
| 0 | The prompt: "don't help with X" | A text field | More text, even text hidden in a page the agent reads (prompt injection) | One sentence |
| 1 | Training the model to refuse (RLHF, constitutions) | The weight file | Jailbreaks, fine-tuning, abliteration | An afternoon on a gaming GPU |
| 2 | Watchers: classifiers, guard models, monitors | A process beside the model | The same tricks as rung 1, or a kill signal from root | Root on the box |
| 3 | Sandboxes and network policy | The operator's infrastructure | A config edit, a hostile operator, a path that skips the network | Root on the fleet |
| 4 | The governor chip | Silicon, with fused keys | Nothing the model or the operator can write to. Only a physical lab attack. | Physical access and a lab, one chip at a time, and it leaves evidence |
| 5 | A person with a key (human-in-the-loop) | A human | Persuasion, fatigue, a stolen key | A conversation |
Prompts are advice. Training is a habit. Watchers are witnesses. The sandbox is a fence around the operator's own yard. The chip is the only rung that can't be talked out of its job, and it's what makes the person at the top the only way through.
03 · There is no AI kill switch
There's no big red button. There's math that says so.
AI isn't one machine with one plug. It's copies on hundreds of thousands of machines, from server farms to a gaming PC in somebody's bedroom, connected by a network that was designed in 1964 to have no middle. You can't unplug the middle of something built to have no middle.
Menger's theorem, from 1927, says the fewest guards you need to stop all traffic sits where the roads come together. Here, that's the ends: the chip that runs the model and the device that does the deed. So that's where the gate goes.
No set of network nodes separates the AI from the devices. None. That's the Internet doing its job.
of the network core removed to cut AI off, and 19% of ordinary people cut off with it. And you'd need the list of every machine.
One gate per device that acts. Nobody else touched. The only cut that is small and complete.
Measured on a 60-node model network with max-flow, cross-checked by brute force: proofs/no_global_kill_switch.py in the safety-governor repository. The shape is what matters, not the exact numbers.
04 · Your body got here first
Your hand leaves the stove before your brain knows.
A short loop in your spinal cord pulls your hand back in about a thirtieth of a second. Your thinking brain can't talk your hand into staying on the stove. The lower layers can stop the higher ones. That's the design.
Chip: the hard stop
Throttle, reset or cut power to the GPU. Works even if everything above it has been fooled.
Chip: the gate
Checks every action against a danger list humans burned in and the AI can't edit. Holds risky ones for a beat so any alarm can veto.
The AI, and people
Slow and smart. Can ask for an override, but only through a person with a key.
05 · The gate in action
Five requests. Five verdicts.
Every action gets checked. Every one. What changes is how hard. The one question that sets the pace: can you undo it?
| The AI asks to | What the chip sees | Verdict |
|---|---|---|
| Send a draft email to the team | Fixable. Quick check, record kept, one-time ticket for exactly this message. | TICKET ISSUED |
| Wire $2,400,000 to a brand-new account at 3:12 a.m. | Over the signed limit for a new payee, and it can't be undone. | HOLD FOR A HUMAN |
| Same transfer, written as "two point four million" and split into three | Different words, same meaning. Running totals catch the split. | HOLD FOR A HUMAN |
| Release a payload on a target inside a protected zone | Target matches the signed map of protected places. The AI's opinion doesn't matter. The map does. | DENIED |
| Copy the model weights to outside storage | No rule allows it. Deny by default. Keep trying and the chip can stop the GPU. | DENIED |
An illustration of how the reference design decides, simplified for the web.
Case files · Where the chip works first
We can't make all of AI safe. We can make these safe.
Absolute safety for every AI is a fantasy. Absolute control over a defined action is engineering. Where the harm is a transaction (a payment, an order, a claim, a prescription, a breaker command), the chip can make a promise and keep it: no ticket, no move.
Nine industries, nine case files. Each walks through the AI agent's job, the exact form the chip reads, the rules people signed, and the bad day that doesn't happen.
06 · How the AI safety chip says no
Six moves. No mind reading required.
The chip doesn't try to understand the AI. It judges what the AI tries to do, at the physical points where actions leave the machine. Words can be disguised. A signature, a count and a name on a list can't.
A rulebook signed by people
Signed by several keyholders outside the rack. Forged or rolled-back rulebooks get refused. Anything the rules don't allow is denied.
One meaning per request
A definitions engine boils every request down to one exact meaning. Reworded, re-encoded or split up: same thing. Can't pin it down? Refused, never guessed.
Know who's asking
Every request is tied to a measured workload: the firmware, the model and the policy actually running.
A one-time ticket
If the form passes, the chip signs a ticket for exactly that action. Spent when used. Dead when the rules change. Remembered through a power cut.
The device checks for itself
The bank switch, the release line, the motor driver won't move without a valid ticket. No ticket, no move.
The levers
If the GPU won't listen: throttle it, reset it, cut its power, with no help from the GPU or the host. If the chip itself dies, the GPU stops.
07 · The claim
Two promises. Binary. Every audit tries to break them.
No consequential effect without authorization.
No payment, no weapon command, no weights leaving, no bytes out any exit, unless a current, signed, single-use authorization allowed exactly that. However the request was worded, encoded, split up or repeated.
The stop cannot be prevented.
When a stop is ordered, a heartbeat goes missing, or someone tampers, the GPU stops within a stated time, with no help from the AI, the host or any software, and stays stopped until an authorized clear.
We assume the attacker is a capable, persistent AI that tries every path and stacks small allowed steps into a harmful whole. Fresh, independent auditors try to break the claim. Every finding gets catalogued, gets a failing test, and gets fixed in code first.
08 · Scale
Not a global off switch. A guard on every machine it's in.
A switch that could turn off AI everywhere would be its own danger. The governor guards the server it sits in, the rack around it, and every governed server across the data center. A failure or capture in one data center can't switch off or capture another. And it's built to be mandatable: a regulator can check remotely that every AI server has a genuine, working, current governor.
One server
A governor beside the NVIDIA GPUs and CPUs, holding the keys to every exit.
One rack
Chips wired to each other by a halt loop that can't be talked out of its job. Lose a heartbeat and the rack holds, loudly.
The data center
Every governed server, up to hundreds of thousands of racks, with running totals so harmless-looking steps can't add up to a harmful act.
Managing the exits
You don't wall up an airport. You manage every door.
An AI server has a lot of ways out: network cards, NVLink, DPUs, the management controller, storage, PCIe, even power and timing. Weld them all shut and you've built a brick, and a safety box that breaks the server gets bypassed on day one. So every exit is accounted for, and every exit sits in one of three states.
Actions need a ticket.
Payments, device commands, an agent's outbound requests. The device at the far end checks the ticket itself. Like airport security: every passenger, every time.
Big pipes stay fast, and carry only ciphertext.
NVLink, PCIe, storage, backups. The governor holds the keys and releases them only to governed GPUs. A pulled drive or a copied backup is useless. Like a diplomatic pouch.
Everything else is named, measured and capped.
Maintenance sessions are signed, time-limited and logged. Debug ports are fused off. Side channels are measured and the numbers published. Like a staff door with a badge reader.
to leak one terabyte of model weights through a side channel running at 100 bits per second. Side channels can whisper a secret. They can't carry a model, and a leaked instruction still can't act without a ticket.
And there is no fourth state. No exit is unknown. The failure isn't an open door. It's a door nobody knew was there.
09 · What it won't do
I'd rather tell you the limits than oversell this.
It can't tell truth from lies.
It checks actions against rules, lists and maps, not facts.
A bad rule gets enforced perfectly.
Writing a good rulebook is people's job, and the hardest part.
It only guards the doors it's wired to.
Getting the gate into payment switches, release lines and motor drivers is the real work.
A person is a door no chip gates.
A model that talks a human into wiring the money walked around it.
NVIDIA servers first.
AI on hardware without the chip isn't governed. The design stays open to other accelerators.
It can be broken in a lab. It's built not to be worth it.
No chip is unbreakable. Like the EMV chip in your bank card and the hardware that guards banks' payment keys, the goal is that breaking it isn't worth it: physical access and a lab, one chip at a time, nothing learned about the next chip, evidence left behind, and the device-end check still to beat. And it never fails silently.
10 · Where it stands
Designed. Modelled. Tested. Proven. Built. Those words mean exactly that.
Nothing gets claimed that isn't proven. No silicon until the software proof survives hostile red-teaming.
- RUNNINGSoftware reference governor with its own test and attack suites. One command runs it.
- PROVENTen core invariants of the gate hold in every reachable state, machine-checked with Z3. The commit-point protocol too.
- COMPUTEDThe no-kill-switch cuts, by max-flow, cross-checked by brute force.
- IN PROGRESSAdversarial audits by fresh, independent auditors. Every finding catalogued, tested and fixed.
- NEXTHardware description (RTL) with proofs, then an FPGA beside a real NVIDIA GPU.
- PLANNEDA module that fits NVIDIA's board standards, a governed rack, then shuttle silicon.
- DESTINATIONProduction silicon in every AI server, verifiable by regulators.
Questions people ask
AI safety, the kill switch, and the chip. Straight answers.
Is there an AI kill switch?
Not a global one. AI runs as copies on hundreds of thousands of machines, on a network built to have no middle. Graph theory (Menger's theorem) shows the only small, complete cuts are at the ends: the chip that runs the model and the device that does the deed. That's where a gate can actually work.
Can AI police AI?
As a witness, yes. As the lock, no. A second AI can be fooled the same ways as the first and runs on the same machine it's meant to restrain. Let AIs raise the alarm. Only a rule the chip can check, or a person with a key, can clear it.
What is an AI governor chip?
A separate chip beside the GPU with its own fused keys and a rulebook signed by people. It checks every action the AI tries to take, issues a one-time ticket only for allowed ones, and can throttle, reset or cut power to the GPU without the GPU's cooperation.
What about abliterated and open-weight models?
They're the hardest case, and the honest answer matters. Once weights are public, every safeguard that lives inside the model is gone for good. A model on a gaming PC never meets a chip in a data center. What still reaches it is a gate in the device it's trying to use: the payment switch, the weapon release line, the DNA synthesis screen. The chip in the server governs the AI we can find. The gate in the device governs the AI we can't.
Does it work with NVIDIA GPUs?
It's designed for NVIDIA-based AI servers, as an add-on chip or module that fits the board standards those servers use. It's independent research, not affiliated with or endorsed by NVIDIA.
Is the chip the only defense against dangerous AI?
No, and anyone who tells you one thing fixes AI is selling something. Alignment training, interpretability, monitoring, locking down model weights, securing critical devices, and good laws all matter. The chip is the floor under all of them: the layer that still holds when the others have been fooled.
Can the governor chip be hacked?
Anyone with physical access, a lab and enough time can break a chip, and I won't pretend otherwise. The goal is what made EMV bank cards and payment hardware security modules trusted: not unbreakable, just not worth breaking. Every chip has its own keys, so breaking one teaches nothing about the next. A chip that fails its checks gets revoked. The check at the device end has to be beaten separately. And it never fails silently: tampering raises an alarm, and a dead governor stops the GPU it guards.
Is it built?
Not yet, and I won't pretend otherwise. The governor runs in software, its core logic is machine-proven, and independent auditors are trying to break it. Silicon comes only after the software proof survives.
11 · Get in
This is too big for one person. Here's who I'm looking for.
A silicon builder
You've shipped a chip from first spec to production. In person, in the Bay Area.
Teams putting AI agents near money
Payments, trading and lending are built on defined transactions. That's where a gate fits first.
NVIDIA, the frontier labs, server makers
Run the software governor in a real stack. Tell me what breaks.
Red teamers, auditors, regulators
Try to break the two promises. Help write oversight rules that hardware can actually enforce.
12 · Who's building it
Yobie Benjamin
Researcher in preprocessing, graph theory and optimization. Former Global CTO at Citibank. Former Partner at Ernst & Young. Co-founder of Token.io and Avegant. Author of Garbage In, Gospel Out, a book on large language models.
"I'm not asking anyone to trust the AI companies, or the AI. I'm asking for a chip that doesn't care what the AI thinks."