Evidence-Triggered Belief Revision
Compare predictions with results and update the assumptions that failed
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 5
- Confidence
- 95%
Asked how he moved from comparing Trump with 'America's Hitler' to becoming his vice president, Vance points to predictions he believes were falsified. He expected Trump to fail as president and trusted major American institutions to function competently; he later concluded that both judgements were wrong. He illustrates the process by rereading his 2016 appeal to military leaders as credible authorities and saying he is now embarrassed by that assumption. The reusable mechanism is not the political conclusion itself. It is to preserve the original forecast, compare it with outcomes, locate the assumption that produced the error, and update that specific belief. A disciplined revision also states its limits: changed evidence about one person or institution does not establish that every later claim is true or that scrutiny should stop.
Origin
Extracted from The Diary of a CEO
Core principles
- 01Predictions should be compared with observed outcomes
- 02A failed forecast may expose a failed underlying assumption
- 03Updating one judgement does not require endorsing everything else
- 04Publicly naming an error strengthens accountability
How to run it
- 1
Capture the original forecast
Write what you expect to happen and why before the result is known. Preserve the important assumptions and the evidence available at the time.
Pro tip Use dated notes so hindsight cannot silently rewrite the prediction.
Watch out A vague forecast is difficult to falsify and easy to rationalize.
- 2
Define revision evidence
Specify which observable results would weaken or overturn the view. Distinguish disconfirming evidence from outcomes that are merely uncomfortable.
Pro tip Set more than one indicator when the judgement has several parts.
Watch out Do not move the standard after seeing the result.
- 3
Compare forecast and result
Review what happened against what was predicted, including misses that cut against your identity or loyalties. Record uncertainty and competing explanations.
Pro tip Ask a person who disagreed with you to identify the clearest miss.
Watch out One outcome may be noisy or too early to justify a broad conclusion.
- 4
Locate the failed assumption
Identify whether the error came from bad data, misplaced trust, an incorrect mechanism, or an unexamined value judgement. Update the cause of the error rather than only the surface prediction.
Pro tip Finish the sentence: 'I expected X because I assumed Y.'
Watch out Replacing one blanket distrust with another repeats the same reasoning failure.
- 5
Revise with boundaries
State the narrowest new belief the evidence supports and what remains unresolved. Continue testing future claims rather than treating the revision as permanent allegiance.
Pro tip Publish a short before-and-after explanation when others relied on the original view.
Watch out Admitting one mistake does not validate every position held by the person who proved you wrong.
In the wild
Vance says he expected Trump to be a failed president and believed established institutions and military leaders were fundamentally functioning. Looking back, he says subsequent events changed those judgements and made him view Trump's conflict with institutional experts differently.
→ He describes the revised assessment as the basis for changing his political support; the interview presents his interpretation rather than an independent evaluation of the presidency.
In this illustrative example, a founder predicts that lowering price will increase paid conversions because prospects describe the product as expensive. The test raises trials but not conversions. Interviews reveal that unclear onboarding, not price, was the binding issue, so the founder restores the price and changes onboarding rather than declaring customers irrational.
→ The failed prediction updates the causal assumption and produces a more targeted experiment.
Common mistakes
Rewriting the old prediction
Softening a forecast after the outcome prevents an honest comparison and protects the failed assumption.
Overcorrecting into loyalty
Evidence against one criticism does not justify accepting every future claim from the same person or institution.
Updating from one anecdote
A broad belief should not be overturned by a vivid example when stronger and more representative evidence is available.
Is it for you?
Best for
It is best for forecasts, strategic judgements, and opinions whose assumptions can be compared with later evidence.
Not ideal for
It is not ideal when evidence is unavailable, the outcome is still unfolding, or the decision involves values rather than an empirical prediction.
From the transcript
“you always have to be able to acknowledge when you're right and when you're wrong”
“I thought Donald Trump would be a failed president if he got elected. He was not.”
“I thought that America's institutions were fundamentally functioning. They were not.”
From the episode
Vice President JD Vance: No One Saw This Coming, The Ceasefire Is Real!