All-Sides Bias Audit
Test every relevant angle before letting feelings drive a factual decision
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 6
- Confidence
- 99%
Tyson describes scientific thinking as doing whatever is necessary to avoid fooling yourself about what is or is not true. His version replaces the demand to inspect both sides with an audit of all relevant sides: variables, causal alternatives, source biases, and sensitivity to changed assumptions. The process begins by separating a felt reaction from the factual claim beneath it. It then looks for measurements capable of contradicting the initial impression and asks whether the conclusion survives. Tyson uses fear of crime as an example: repeated exposure to vivid footage can raise perceived danger even when longer-run crime data points in another direction. The framework does not dismiss emotion; it limits emotion's authority when a shared decision depends on objective evidence.
Origin
Tyson applies his account of the scientific method to public reasoning, contrasting recurring reports of feeling less safe with the longer-run crime trend he says the data showed.
Core principles
- 01Most consequential questions have more than two sides
- 02Feelings can signal concern without establishing objective conditions
- 03A conclusion should survive attempts to disprove it
- 04Evidence must be checked for sensitivity and bias
How to run it
- 1
Separate feeling from claim
Write down the emotional response and the objective proposition it appears to imply as two different statements.
Pro tip Treat the feeling as real while leaving the proposition open to testing.
Watch out Neither dismissing emotion nor treating it as proof completes the analysis.
- 2
Map all sides
List relevant explanations, affected groups, variables, and plausible alternatives rather than forcing a binary contest.
Pro tip Ask what would change if a key assumption moved.
Watch out Adding weak possibilities for appearance's sake is not the same as finding material alternatives.
- 3
Audit bias channels
Identify how vivid media, incentives, sampling, identity, recency, or your preferred conclusion could distort judgment.
Pro tip Name the bias mechanism before searching for confirming evidence.
Watch out Knowing the name of a bias does not prove you have removed it.
- 4
Seek disconfirming measurements
Find data or observations that could make your current view fail, then compare them with the evidence supporting it.
Pro tip Prefer broader trends and direct measures over memorable examples when the question concerns prevalence.
Watch out A single statistic can also mislead if its definition or time window is wrong.
- 5
Stress-test the conclusion
Change reasonable assumptions and assess whether the result remains stable enough to guide action.
Pro tip Record which uncertainty would actually reverse the decision.
Watch out Do not claim certainty beyond what the evidence supports.
- 6
Act on surviving evidence
Make the factual part of the decision from the strongest supported conclusion while addressing values openly and separately.
Pro tip Revisit the decision when a named reversal condition occurs.
Watch out Evidence cannot decide which values a community should hold.
In the wild
Tyson cites polling in which Americans repeatedly said their communities felt more dangerous than the previous year, then contrasts that perception with a long-term crime rate he says generally declined. He suggests vivid local-news footage as one possible source of the mismatch.
→ The comparison illustrates why a felt trend should be checked against directly relevant data.
Illustrative example: a team feels customer fraud is suddenly surging after three severe cases. It separates severity from prevalence, checks the full rate over time, audits reporting changes, tests alternative explanations, and then targets the verified pattern rather than redesigning the product around three vivid incidents.
→ The intervention matches measured risk while preserving attention to the serious cases that triggered concern.
Common mistakes
Forcing two sides
A binary frame can hide additional variables, affected groups, and causal explanations that change the decision.
Using feelings as prevalence data
Vivid experience can matter greatly without accurately measuring how often an event occurs.
Calling one data point objective truth
Definitions, sampling, time windows, and sensitivity still need examination before a statistic can support the conclusion.
Is it for you?
Best for
It is best for research, policy, organizational decisions, and disputed claims where evidence can be gathered and compared.
Not ideal for
It is not ideal for choices based primarily on personal taste, values, or relationships where no objective answer exists.
From the transcript
“No, I looked at all sides.”
“figure out all the ways you could bias yourself”
“do whatever it takes to not fool yourself”
From the episode
Neil deGrasse Tyson: DO THIS Every Morning To Find Happiness & Meaning In Your Life