Sound-Good versus Work Test
Judge interventions by outcomes, not by how virtuous they appear
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 5
- Confidence
- 89%
The Sound-Good versus Work Test separates moral presentation from practical effect. State the concrete result an intervention is meant to produce, establish a baseline and select an observable measure before launch. Then identify foreseeable harms, including messages or incentives that may undermine the intended beneficiaries. Run the intervention on a bounded scale and compare outcomes rather than applause, participation or online approval. Kisin develops this distinction while discussing well-intentioned support programs and symbolic social-media activity, arguing that some actions can feel progressive while failing to help. The test does not assume such programs fail; it requires evidence before their intentions are treated as proof of effectiveness.
Origin
Extracted from The Diary of a CEO
Core principles
- 01Good intentions do not guarantee good outcomes
- 02Visible virtue is not a substitute for impact
- 03Real-world feedback should decide whether an intervention continues
- 04Counterproductive effects count even when the goal is admirable
How to run it
- 1
Specify the result
Translate the admirable intention into a concrete change for the people affected.
Pro tip Ask what would be different if the intervention genuinely worked.
- 2
Set the evidence
Choose a baseline, a success measure and a review date before implementation.
Watch out Engagement and praise may measure visibility rather than impact.
- 3
Map side effects
List ways the intervention could stigmatize, displace, discourage or otherwise harm its intended beneficiaries.
Pro tip Include feedback from the people the intervention is meant to help.
- 4
Test and compare
Run a bounded version and compare the measured result with the baseline and a credible alternative.
Watch out Do not redefine success after seeing weak results.
- 5
Keep, change or stop
Scale what works, redesign mixed results and end interventions whose benefits do not justify their costs.
Pro tip Record the decision so symbolic pressure does not quietly revive a failed approach.
In the wild
Illustrative example: a city tests a targeted grant with one cohort and tracks business starts, twelve-month survival and participant confidence against a matched support-only cohort. Interviews also check whether the eligibility framing made applicants feel underestimated.
→ The city can retain, redesign or stop the grant using outcomes rather than its benevolent intent.
Common mistakes
Treating intentions as evidence
A compassionate purpose explains why to test an intervention, not whether it succeeded.
Measuring only public approval
Likes, praise and participation can rise while the target outcome stays flat or worsens.
Is it for you?
Best for
It is best for policies, workplace initiatives and social programs with measurable outcomes.
Not ideal for
It is not ideal for duties whose value cannot ethically depend only on measurable utility.
From the transcript
“We have replaced things that work with things that sound good.”
“They don't actually work very well.”
From the episode
The Anti-Woke Expert: “USA & UK Are Brainwashed!”