The Unknown-to-Experiment Recipe
Turn an unanswered question into data that could distinguish explanations
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 6
- Confidence
- 97%
Tyson describes confronting the unknown as writing a recipe for more data. Instead of filling a gap with a preferred answer, define what observation would distinguish the live explanations and build or select a tool capable of making that measurement. He applies this logic to cosmology, UFO claims, and reported shared psychedelic experiences: telescopes can probe dark energy, physical evidence could settle claims of recovered aliens, and independently matching information could test whether two subjective experiences share more than a common brain response. The mechanism is disciplined uncertainty. A question becomes productive when it names the unknown, specifies rival explanations, identifies a discriminating observation, and produces repeatable evidence that other people can inspect. Results narrow the claim rather than licensing certainty beyond what was measured.
Origin
Extracted from The Diary of a CEO
Core principles
- 01Not knowing is an acceptable starting point
- 02A useful question specifies what data would change the answer
- 03Competing explanations need a discriminating test
- 04Repeatable measurements outrank vivid subjective reports
- 05Tools can extend observation beyond unaided senses
How to run it
- 1
Declare the frontier
Describe what is known, what is observed, and the precise gap that remains. Resist supplying an answer simply because uncertainty is uncomfortable.
Pro tip Write the unknown as a question that could receive more than one answer.
Watch out An unexplained observation is not evidence for whichever explanation is most dramatic.
- 2
List competing explanations
Record the plausible mechanisms that could produce the observation. Include ordinary explanations alongside novel ones.
Watch out Do not frame the test so that only your favored explanation can win.
- 3
Specify discriminating data
Ask what result would be more likely under one explanation than another. Define that result before collecting evidence.
Pro tip Prefer observations that could weaken your preferred account as well as support it.
- 4
Extend observation with tools
Choose an instrument, detector, survey, or controlled procedure that can capture the required information beyond unaided perception.
Pro tip Make the output inspectable by people who did not share the original experience.
Watch out A more sophisticated tool does not fix a poorly specified question.
- 5
Test for repeatability
Repeat the measurement or seek an independent result under comparable conditions. Check whether the same pattern survives.
Watch out Agreement among observers is useful, but it does not by itself prove the proposed mechanism.
- 6
Update narrowly
Revise the live explanations according to the result and clearly retain any uncertainty. Design the next data request if more than one explanation remains.
Pro tip State what the experiment did not test.
In the wild
Tyson proposes testing whether one person could obtain information from a reported psychedelic entity that independently matches information obtained by someone else. He presents this as a way to probe the claim rather than accepting that the experience is externally real or dismissing the commonality without investigation.
→ A metaphysical claim is converted into a proposed comparison with observable results.
In an illustrative product use, a team sees falling activation and lists three possible causes: confusing setup, poor traffic fit, or a technical error. It defines a separate observable prediction for each, instruments the funnel, and interviews failed users before choosing a redesign.
→ The team gathers evidence that distinguishes causes instead of building from the loudest opinion.
Common mistakes
Treating unexplained as extraordinary
Tyson accepts that some sightings remain unidentified, but he does not treat that status as proof of alien origin.
Collecting data with no decision rule
More observations are useful only when the test says how different results would change the competing explanations.
Claiming certainty from one test
A result may narrow the possibilities without establishing a complete account of the underlying reality.
Is it for you?
Best for
It is best for research, investigations, and product questions where several explanations fit the current observations.
Not ideal for
It is not ideal when an experiment would be unethical, impossible, or too dangerous to justify.
From the transcript
“when you confront the unknown, you write the recipe for more data”
“let's do experiments on that”
“they're answerable using the methods and tools of science for what is repeatable”
From the episode
Neil deGrasse Tyson On Aliens, Simulation Theory, and What Happens Inside A Black Hole