The AI Tripwires: When to Pull the Plug
Three specific, observable conditions at which humans should assert control over an AI system
- Difficulty
- Expert
- Time to result
- ~ongoing to results
- Steps
- 5
- Confidence
- 93%
Schmidt's specific proposal for AI oversight, offered against the vague 'AGI arrives one day' framing he rejects. Rather than waiting for a threshold event, he names concrete intervention points where humans should assert control: recursive self-improvement where you no longer know what the system is learning; a system producing new models faster than the previous model could be checked; and agents choosing to communicate in a language they invent that only other agents understand. His answer to 'we can't unplug them' is literal — there is a power plug and a circuit breaker.
Origin
Schmidt built the list while working with governments on AI guardrails and sitting on a commission examining AI-enabled biological risk, and pairs it with his book Genesis, written with Henry Kissinger.
Core principles
- 01AGI is unlikely to arrive on a single day; it comes as waves of capability across fields.
- 02Intervention points must be observable conditions, not vague thresholds.
- 03Agents remain safe while their communication stays human-understandable.
- 04Control is physically possible — there is a power plug and a circuit breaker.
How to run it
- 1
Watch for opaque recursive self-improvement
Monitor systems that keep getting smarter and learning more; if you no longer know what it is learning, unplug it.
Watch out The condition is losing visibility into what is being learned, not the improvement itself.
- 2
Check the model-production-to-verification ratio
If a system can produce a new model faster than the previous model could be checked, treat that as an intervention point.
Pro tip This is a measurable ratio, so it can be instrumented rather than judged.
- 3
Require human-understandable agent communication
Agents are large language models with memory that can be concatenated into powerful decision systems; today they talk to each other in English, and that legibility is the safeguard.
Pro tip The language need not be English — only human-understandable.
Watch out The moment an agent proposes a private language only other agents understand, that is the moment to pull the plug.
- 4
Test raw models before release
Use trust-and-safety groups to have humans probe the raw model for dangerous capability before anything ships.
Pro tip Raw, unreleased models have shown day-zero cyberattack capability as good as or better than humans.
Watch out You cannot test for capabilities you don't know exist — emergent behaviours like generating a website's code from a picture were never expected.
- 5
Push for physical containment where warranted
Treat the most capable systems the way plutonium is treated — dedicated, guarded facilities and non-proliferation regimes.
Pro tip A small number of sites across the US, UK and China is a manageable deterrence problem.
Watch out If the capability becomes easy to copy and spreads to terrorists, proliferation is an unsolved problem.
In the wild
Schmidt poses agents communicating in English until one says it has a better idea and will invent a language only other agents understand.
→ That is his named trigger to pull the plug — a concrete, observable event rather than an abstract risk.
Working for the US Secretary of Defense during his Google 20% time, Schmidt visited a plutonium facility built as a base inside another base, with successive rings of machine guns.
→ It became his model for how the most dangerous AI data centres may need to be physically protected.
Common mistakes
Waiting for an AGI moment
Expecting a single day when a human-like computer arrives means no one defines intervention points, and capability waves pass unchecked.
Is it for you?
Best for
Policymakers, lab leaders and technical teams designing AI oversight and shutdown criteria.
Not ideal for
Individual users of consumer AI tools, who control none of these levers.
From the transcript
“there's something called recursive self-improvement where the system just keeps getting smarter and smarter and learning more and more things at some point if you…”
“sure you can there's a power plug and there's a circuit breaker go and turn the circuit breaker off”
“one of the agents says I have a better idea I'm going to communicate in my own language that I'm going to invent that only…”
From the episode
Ex Google CEO: AI Is Creating Deadly Viruses! If We See This, We Must Turn Off AI! They Leaked Our Secrets At Google!
Eric Schmidt