Audience Evidence Loop
Combine comments, behaviour, and experiments to improve what you create
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 6
- Confidence
- 99%
Maini treats content development as a two-way conversation with the audience. Explicit feedback includes direct comments about what worked or what viewers want changed. Implicit feedback includes likes, minutes watched, and the moments where viewers leave. The method combines both: locate a behavioural signal, form a specific hypothesis about what caused it, change one meaningful element, and compare the next result. When an idea has no precedent—his example is a genuinely new product or video concept—begin with a small experiment and let observation follow. The loop remains audience-first without becoming audience-controlled: viewers may not be able to describe an innovation before seeing it, and one metric cannot explain itself. The creator's job is to respect attention, test interpretations, and remain willing to abandon an ego-attached hypothesis when behaviour contradicts it.
Origin
Extracted from The Diary of a CEO. Arun Maini describes using explicit comments and implicit watch behaviour to understand viewers, while Steven Bartlett summarises invention as experiment followed by analytical observation.
Core principles
- 01The viewer or customer is the recipient of the value
- 02Stated feedback and observed behaviour answer different questions
- 03Retention drops are signals to investigate, not automatic instructions
- 04Novel ideas begin as experiments when no historical data exists
- 05Iteration requires low ego about an initial hypothesis
How to run it
- 1
Define the Audience Promise
State who the content serves and what value their time should receive. Use that promise to distinguish useful evidence from unrelated requests.
Pro tip Write the promise from the viewer's perspective.
- 2
Collect Explicit Feedback
Review comments and direct requests for recurring praise, confusion, or desired changes. Preserve the underlying need rather than obeying every proposed solution.
Pro tip Cluster repeated comments before acting.
Watch out A loud comment is not automatically representative.
- 3
Read Implicit Behaviour
Inspect likes, watch time, completion, and drop-off points. Mark moments where behaviour changes materially.
Pro tip Pair the graph with the exact sentence or event at that moment.
Watch out A nearby event may correlate with a drop without causing it.
- 4
Form One Hypothesis
Explain what feature may have produced the observed response and why. Make the claim narrow enough to test in the next piece.
Pro tip Separate the signal from your interpretation of it.
- 5
Run the Experiment
Change one meaningful element or trial a new idea at bounded cost. Publish enough to obtain a comparable audience response.
Pro tip Use experiments when no historical data can answer the question.
- 6
Compare and Iterate
Compare the result with the relevant baseline, then keep, revise, or remove the change. Update the next hypothesis instead of defending the original idea.
Pro tip Look for repeated evidence across more than one release.
Watch out Do not optimise retention at the expense of truth or promised value.
In the wild
Maini says he reviews what percentage of a video people watch and where they leave. When a particular sentence coincides with a drop, he investigates what about it may have lost viewers and adjusts later work rather than simply increasing output.
→ Audience behaviour becomes evidence for improving both the channel and his presentation.
Illustrative example: a show sees repeated early drop-offs after long guest biographies. The producer hypothesises that listeners want the episode's value sooner, tests a shorter biography on three episodes, and compares first-five-minute retention before adopting the change.
→ A behavioural signal is tested rather than treated as a self-explanatory command.
Common mistakes
Creating Only for Yourself
Maini rejects the idea that passion alone guarantees an audience when the viewer's time and needs are ignored.
Treating the Graph as an Explanation
A drop-off identifies where to investigate; it does not by itself prove why people left.
Asking Audiences to Invent the Future
Maini notes that people may not request a novel idea before experiencing it, so invention still needs a trial.
Is it for you?
Best for
It is best for creators and product teams with recurring output, behavioural data, and enough volume to compare experiments responsibly.
Not ideal for
It is not ideal for blindly maximising one metric, copying noisy comments, or treating correlation at a drop-off point as proof of causation.
From the transcript
“there's explicit and there's implicit feedback”
“it's got to start with an experiment and then go into analytical observation”
From the episode
How To Build A Following Of 10 Million +: Mrwhosetheboss