Twenty-Percent Failure Rule
Use some failed tests as evidence that experimentation is ambitious enough
- Difficulty
- Moderate
- Time to result
- ~ongoing to results
- Steps
- 5
- Confidence
- 97%
Isaacson describes Musk's view that a team failing none of its experiments is probably not taking enough risk. Musk's quoted benchmark is roughly 20 percent, but the useful mechanism is not the number by itself. Teams define an unknown, contain the possible downside, run an informative test, inspect what went wrong, and use the result to improve the next attempt. Isaacson uses the first two Starship test launches as Musk's examples: despite explosions, Musk regarded them as useful because they exposed problems that could be fixed. This is a portfolio signal for experimentation, not a universal safety target. The rule only transfers responsibly when failures are bounded and learning is captured; it should not normalize preventable harm or reckless testing.
Origin
Isaacson says Musk considered two explosive Starship test launches successful experiments because each revealed problems the team could fix.
Core principles
- 01Experiments contain unknowns
- 02A zero-failure record can indicate risk avoidance
- 03Failure is useful only when it produces learning
- 04The acceptable failure rate depends on consequences
How to run it
- 1
Name the unknown
State what the team does not yet know and what result would reduce that uncertainty.
Pro tip Test one dominant uncertainty where possible.
Watch out Do not call routine execution an experiment merely to excuse poor quality.
- 2
Bound the downside
Set limits on cost, safety exposure, reversibility, and blast radius before running the test.
Pro tip Choose the smallest test that can still produce useful evidence.
Watch out Some domains should target far below a 20 percent failure rate.
- 3
Run the test
Execute under observable conditions and preserve the data needed to explain the outcome.
Pro tip Define success and failure before seeing the result.
- 4
Extract the failure
Identify what went wrong and distinguish useful uncertainty from avoidable negligence.
Pro tip Write the change the next test will make.
Watch out An unexplained failure is not automatically productive.
- 5
Review the portfolio
If every bounded experiment succeeds, ask whether the team is testing meaningful unknowns; if too many fail, improve preparation and controls.
Pro tip Use the benchmark diagnostically rather than as a quota.
Watch out Never manufacture failures to reach a percentage.
In the wild
Isaacson notes that headlines described two Starship launches as explosions. He says Musk instead treated them as experiments that revealed what went wrong, allowing the team to fix problems before another attempt.
→ Failure data became input to another design iteration rather than the end of the programme.
Common mistakes
Turning the benchmark into a quota
The transcript presents 20 percent as a risk signal, not a requirement to cause failures.
Ignoring consequence severity
A tolerable software prototype failure and a safety-critical failure do not carry the same cost.
Failing without learning
A failed test only advances the work when the team captures evidence and changes the next attempt.
Is it for you?
Best for
It is best for bounded research and product experiments where failures are observable, reversible, and safe.
Not ideal for
It is not ideal for medical, safety-critical, irreversible, or high-blast-radius decisions where failure could seriously harm people.
From the transcript
“if you're not failing 20% of the time, you're not risking enough”
“an experiment involves the unknown and taking a risk”
From the episode
The Man Who Followed Elon Musk Everywhere: "Elon's Dad Abused Him, His Trans Child Disowned Him, And Here Are His Secrets For Success!" Walter Isaacson