MISSION · Data and probability
Confidence Intervals
What the margin of error in a poll actually claims
LEVEL 1 · EXPLORE
Make the idea move
Change the controls and watch what happens. Follow your curiosity and see what you notice.
Loading the activity…
LEVEL 2 · LEARN
Unlock the explanation
Open the chapters in any order. Try the examples as you go.
The idea in one sentence
A sample gives one number; a confidence interval gives the range of population values that number is consistent with, plus how often such ranges are right.
\[\text{estimate} \pm \text{margin of error}\]Why a single number is never enough
A poll finds 52% support. The true figure across the population is almost certainly not exactly 52% — a different random sample would have given something slightly different.
So reporting “52%” alone overstates what was learned. Reporting “52% ± 3 points” states the finding and its precision together, and that second part is what makes the first usable. In this case the interval runs from 49% to 55%, which includes 50 — so the poll does not actually establish a majority, however the headline reads.
Where the margin comes from
Sample means vary, and their variation is measurable. The standard error describes the spread of the estimate itself:
\[SE = \frac{\sigma}{\sqrt{n}}\]and for a 95% interval you go about 1.96 standard errors either side of the estimate.
The \(\sqrt{n}\) is the important part. To halve your margin of error you need four times the data, not twice. That’s why polls cluster around 1,000 people: it buys about ±3 points, and pushing to ±1.5 would cost four times as much for a modest gain.
What “95% confident” means
This is the part that gets misstated constantly, including by people who should know better.
The correct reading: if you repeated the whole sampling procedure many times, 95% of the intervals produced would contain the true value. The confidence is in the method, across repetitions.
What it does not mean: that there is a 95% chance the true value lies in this particular interval. The true value is a fixed number, not a random one. This interval either contains it or doesn’t — you simply don’t know which, and one interval in twenty misses entirely.
That distinction matters in practice. Twenty studies, each perfectly conducted, will on average produce one interval that excludes the truth. Nothing went wrong when that happens; it’s what 95% means.
The trade-off
Want more confidence? Widen the interval. A 99% interval catches the truth more often precisely because it claims less about where the truth is.
“Between 0% and 100%” is a 100% confidence interval, and it is useless. Certainty and precision pull against each other, and choosing 95% is a convention balancing them, not a discovery.
Common mix-ups
- “95% chance the truth is in this interval.” The probability belongs to the procedure.
- Assuming a wider interval is worse. It’s more cautious, which may be appropriate.
- Ignoring bias. A confidence interval accounts for random sampling error only. It says nothing about a badly drawn sample, and a biased poll produces a confident wrong answer.
- Comparing overlapping intervals casually. Overlap does not by itself mean no difference.
Check yourself
A poll of 1,000 finds 47% support with a margin of error of 3 points. Can the campaign claim it is behind?
The interval is 44% to 50%, so support below half is consistent with the data — but so is exactly half. The poll is compatible with being behind or level, and a claim of certainty either way overstates it.
LEVEL 3 · REFLECT
Add this quest to your trail
When you feel ready, mark this idea as explored. You can always come back and try it again.
EXPLORER STRATEGIES