Within-Person Baseline Test
Compare each person with their own norm before inferring an effect.
- Difficulty
- Advanced
- Time to result
- ~months to results
- Steps
- 5
- Confidence
- 95%
Von Hippel explains that comparing one kind of person with another can confound a behavioral analysis because the groups may differ in many unmeasured ways. His preferred approach is within-person: establish what each participant usually does, then compare that person's outcomes on days when the behavior is absent, typical, or greater than usual. Aggregate those individual deviations only after making the self-comparison. In the Whoop example, this means asking how the same person's biomarkers differ across drinking and exercise conditions rather than comparing a person who often combines them with someone who does not. The method improves observational inference but does not establish causality by itself. Results still require robustness checks, physiological interpretation, and independent scientific review.
Origin
Von Hippel describes the method while explaining how the Whoop performance-science team analyzes repeated alcohol, exercise, and biomarker observations. Extracted from The Diary of a CEO.
Core principles
- 01Differences between people can conceal confounding variables.
- 02A person's usual behavior provides a more relevant comparison baseline.
- 03Repeated observations are needed before interpreting a pattern.
- 04An observational pattern remains provisional until it is vetted.
How to run it
- 1
Define the variables
Specify the behavior, time window, outcome, and plausible covariates before inspecting the pattern. Keep measurement definitions consistent across observations.
Pro tip Separate exposure intensity from simple presence or absence where the data support it.
Watch out Do not change definitions after seeing a preferred result without disclosing it.
- 2
Build individual histories
Collect enough repeated observations to characterize each participant's usual behavior and outcome. Exclude people whose records cannot support a meaningful baseline.
Pro tip Check whether logging behavior itself changes over time.
Watch out Self-reported exposures may contain systematic error.
- 3
Normalize to the person
Express each observation relative to that same person's typical level. This limits distortion from stable differences between participants.
Pro tip Preserve the raw values alongside the normalized measures.
Watch out Within-person comparison does not remove time-varying confounding.
- 4
Compare conditions
Compare the participant's outcomes across relevant behavior conditions, then aggregate the within-person differences across the sample.
Pro tip Inspect whether a small subgroup is driving the average.
Watch out Association in observational data is not proof that the behavior caused the outcome.
- 5
Challenge the result
Test alternative explanations, sample choices, timing assumptions, and measurement artifacts. Treat an unreviewed result as provisional until it survives independent scrutiny.
Pro tip State what evidence could overturn the interpretation.
Watch out Do not turn an unpublished pattern into medical guidance.
In the wild
Von Hippel says the team compares the same person's biomarker response on days with different drinking and exercise patterns. He explicitly cautions that the findings discussed are unpublished, may contain an overlooked error, and need scientific vetting.
→ Stable differences between people are reduced, while the resulting association remains appropriately provisional.
Common mistakes
Comparing different kinds of people
A person who exercises when drinking may differ from another person in many ways besides the measured behavior.
Calling association causation
Within-person analysis improves the comparison but does not randomize behavior or eliminate every changing confounder.
Publishing the exciting interpretation first
Von Hippel repeatedly labels the Whoop findings as unvetted and potentially mistaken.
Is it for you?
Best for
Analysts studying repeated behavior and outcome measurements from the same participants over time.
Not ideal for
Sparse one-off surveys, causal claims without adequate design, or medical decisions based only on observational wearable data.
From the transcript
“what I really want to know is what does Steven look like”
“we want to make it all against what you usually do”
From the episode
Behavioural Psychologist: We're Not Having Enough Sex! Fat Makes You Attractive & The Poorer You Are The Friendlier You Are!