Understanding Survivorship Bias
Survivorship bias is the mistake of drawing conclusions only from the cases that made it through, while overlooking the invisible ones that didn't — and recognizing it sharpens every judgement about success, risk, and data.
A complete interactive classroom, not just a preview.
Start when you are ready to enter this Stage's 12 scenes and explore, respond, and learn as you go.
Why do the 'success stories' we see everywhere give us a distorted picture of reality?
- what-is-survivorship-bias
- Drawing conclusions from only the visible 'survivors' while ignoring the invisible cases that were filtered out.
- wwii-bomber-example
- Abraham Wald's insight: damaged bombers that returned mark non-critical areas; the missing bombers reveal critical vulnerabilities.
- missing-data-distortion
- Why a sample of survivors systematically misrepresents the full population it came from.
- everyday-examples
- Startup advice, 'rich person habits', fitness transformations, and stock tips all reflect survivors only.
- spotting-it
- Checklist for noticing when a story, dataset, or claim is built on the visible winners alone.
- mitigation
- Ask 'who is missing?', seek counterexamples, and look at base rates before trusting survivor-based advice.
If we only look at the cases that succeeded, we can learn what works.
Show that successes alone hide the failures; the lesson 'learned' is biased toward traits shared by survivors, not traits that caused survival.
The WWII engineers were right to add armor where returning bombers had bullet holes.
Show that bullet holes on returning bombers mark the non-critical areas; the planes hit in critical spots never came back.
Survivorship bias only matters in military or financial data, not everyday life.
Show that self-help advice, startup tips, and health studies are all shaped by who is visible in the sample.
- formal statistical inference
- selection bias in epidemiology (deep dive)
- publication bias in meta-analysis
- survival analysis / Kaplan-Meier
- Learner explains survivorship bias in their own words using one new example.
- Learner identifies the missing group in a given scenario.
- Learner applies the 'who is missing?' check to a claim they encounter.
- Spot survivorship bias in a real news story, testimonial, or piece of advice and articulate who is missing from the picture.
Curious beginners with no statistics background. Comfortable with everyday reasoning, but unfamiliar with formal sampling concepts.
- 01What Is Survivorship Bias?slideOrientationObserve
Introduce the concept with a concrete hook: we only see the winners, and the losers are invisible.
- Defining survivorship bias
- Why the missing cases matter
- Preview of the WWII bomber story
- 02Spot the Bias: The Bullet-Riddled BomberinteractivePredictionPredict
Learners see a bomber diagram with bullet holes from returning planes and predict where to add armor — before learning the twist.
- Predict where armor is needed
- Reveal the hidden group (planes that didn't return)
- Feel the intuition shift
- 03The Invisible Ones: Drag the Missing CasesinteractiveMisconception repairConstruct
A drag-and-drop puzzle where learners move 'missing' cases (failed startups, dropped stocks, vanished planes) back into the picture to restore the full population.
- Drag the invisible cases into view
- Watch the average and the story change
- Feel what 'full data' looks like
- 04Abraham Wald and the BombersslideModel buildingExplain
Explain the historical episode clearly: Wald's statistical reasoning and why the bullet holes marked the safe zones.
- Wald's insight
- Why the missing bombers are the data
- Counterintuitive but rigorous
- 05The Two Averages SimulatorinteractiveMisconception repairObserve
A simulation showing how 'average' of survivors diverges from 'average' of the full population as the filter strengthens.
- Drag a slider to set the survivor threshold
- Watch the survivor mean vs. full mean diverge
- See bias grow in real time
- 06Survivor Stories MapinteractiveModel buildingObserve
An interactive diagram linking everyday examples (startups, stocks, habits, fitness) to the same underlying bias pattern.
- Five everyday examples
- One shared structure: only winners are visible
- Click each node for a 1-line explanation
- 07Red Team the AdviceinteractiveApplicationApply
Learners are shown real-sounding advice ('Read these 5 books that changed my life') and must drag the right 'missing group' into the picture to weaken the claim.
- Read the claim
- Identify the invisible missing group
- Articulate why the advice is suspect
- 08How to Spot Survivorship BiasslideModel buildingExplain
A short framework: who is missing? what filter did they pass? what would the non-survivors say?
- Three diagnostic questions
- Examples for each
- When to be more skeptical
- 09Spot the Bias in the WildinteractiveApplicationChoose
A fast-paced action game: headlines and testimonials scroll past; the learner taps only the ones that smell like survivorship bias before time runs out.
- React quickly
- Classify 8–10 real-style claims
- Build pattern recognition
- 10Build a Survivorship-Resistant AnalysisinteractiveSynthesisConstruct
A simulation where learners design a small study from scratch and must explicitly account for the missing cases to make their estimate accurate.
- Define a question
- Choose who is in the sample
- Account for missing cases
- Compare biased vs. corrected estimate
- 11Key TakeawaysslideSynthesisExplain
Recap the definition, the bomber story, the everyday examples, and the diagnostic questions in one tight summary.
- Definition in one sentence
- Wald's lesson
- Three diagnostic questions
- Who is missing?
- 12Final Challenge: Audit a HeadlineinteractiveAssessmentApply
An action-style assessment where the learner is given a fresh, plausible headline and must produce a short written audit naming the missing group.
- Read a new claim
- Write who is missing
- Self-check against a rubric
Discussion threads for a Stage aren't available yet.
