Consequence-First Policy Test
Judge a policy by real effects, not identity, intentions, or moral comfort
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 5
- Confidence
- 97%
Kisin argues against evaluating politics mainly through personalities, party identity, or how a policy makes supporters feel. His alternative is to state what a proposal is supposed to achieve, inspect the mechanism and incentives, and compare the observed consequences with the promise. He applies this reasoning to taxes that may encourage wealth creators to leave and climate policies that he claims outsource production without reducing global emissions. Those substantive claims require independent verification; the transferable method is the evaluation discipline. It also explains why he says he would support any leader who adopted policies he believed would work. A complete test should include second-order effects, credible counterfactuals, distributional impacts, rights and ethical constraints, and a willingness to change course when reality contradicts the intended story.
Origin
Extracted from The Diary of a CEO. Konstantin Kisin invokes Thomas Sowell's criticism of replacing what works with what feels good and applies it across tax, energy, and leadership debates.
Core principles
- 01Good intentions do not guarantee useful outcomes
- 02A leader's identity is less important than the effects of the policy
- 03Incentives and second-order effects belong in the evaluation
- 04A policy that fails in practice should not survive on emotional appeal alone
How to run it
- 1
Define the Outcome
Translate the policy's moral or political promise into observable results. Include who should benefit, by how much, and over what period.
Watch out A slogan is not a measurable outcome.
- 2
Trace the Mechanism
Explain how the policy changes incentives, behaviour, resources, or constraints to create the result. Identify assumptions that must hold.
Pro tip Write the causal chain in plain language.
- 3
Count the Consequences
Measure intended results, unintended effects, displaced activity, and who bears the costs. Attribute uncertain forecasts as forecasts rather than facts.
Watch out Do not count visible benefits while ignoring costs moved elsewhere.
- 4
Compare the Alternative
Test the policy against a credible counterfactual, not against doing nothing by default. Include ethical and legal constraints in both cases.
Pro tip Ask what would probably happen without the intervention.
- 5
Update Without Tribalism
Keep, modify, or stop the policy according to evidence rather than the identity of its sponsor. Record what result would trigger another review.
Watch out Outcome focus does not permit violating rights or hiding distributional harm.
In the wild
Kisin and Bartlett argue that raising taxes on mobile high earners may reduce the tax base if entrepreneurs leave. Bartlett cites estimates about a founder's potential capital-gains bill and equivalent average taxpayers; the episode presents these as claims and simple calculations, not a verified causal estimate.
→ The example shows why a policy should be tested against behavioural response and total revenue rather than its stated target alone.
Kisin claims that closing domestic production can shift emissions to countries with dirtier processes and add shipping emissions. Bartlett accepts that outsourcing would not improve the climate outcome if that mechanism is correct; neither speaker claims climate expertise.
→ The example distinguishes reducing a national accounting total from reducing the global harm the policy is meant to address.
Common mistakes
Rewarding Intentions Instead of Results
A policy can express an appealing value while producing effects that undermine its stated purpose.
Ignoring Behavioural Response
People and organisations adapt to incentives, so a static estimate can miss the policy's real effect.
Using Outcomes to Excuse Harm
Practical evaluation must still respect rights, law, ethics, and unequal burdens.
Is it for you?
Best for
It is best for evaluating public policy, organisational rules, and leaders whose proposals carry measurable costs and benefits.
Not ideal for
It is not ideal when measurable outcomes are used to erase legal rights, ethical duties, or harms that cannot be reduced to one metric.
From the transcript
“I am not one of them of any them. I'm just telling you what the policies are”
“replacing what works with what feels good”
“what makes us feel good as opposed to what actually works in practice”
From the episode
The Man Warning The West: Trump Is Changing The World Behind The Scenes!