Experiments and Observational Studies

The idea in one sentence

If you assign the treatment, you have an experiment and can talk about cause; if you only record what people already do, you have an observational study and cannot.

The two kinds

In an observational study you measure without intervening. Survey a thousand people about coffee and sleep quality, and see how the two relate.

In an experiment you decide who gets what. Take a hundred volunteers, give a randomly chosen half two cups of coffee and the other half none, and compare their sleep.

The difference is not the size, the cost, or the care taken. It is whether the researcher assigned the treatment.

Why assignment is the whole story

Suppose coffee drinkers sleep worse. In an observational study, at least three explanations survive:

1. Coffee causes poor sleep.
2. Poor sleep causes coffee drinking — tired people reach for the pot.
3. Something else causes both. Long working hours produce more coffee and less sleep.

That third thing is a confounding variable: something linked to both the treatment and the outcome, which makes it impossible to say which is doing the work. It is the reason correlation does not imply causation, and simply collecting more data does not fix it. A confounded study with a million people is still confounded.

Random assignment dissolves all three at once. If a coin decides who drinks coffee, then working hours, age, stress and everything else — including the confounders nobody thought of — end up spread roughly evenly between the groups. Any remaining difference in sleep has only one source left.

The four things a good experiment has

A control group. Something to compare against. Without it, you learn the treated group’s outcome and nothing about whether the treatment mattered.

Random assignment. Chance decides who goes where. Letting people choose, or assigning the sickest patients to the new drug, reintroduces exactly the confounding you were trying to remove.

Replication. Enough subjects in each group that ordinary variation does not swamp the effect. Two people per group tells you nothing.

Control of other variables. Same schedule, same measurement method, same room. What you cannot hold fixed, randomisation handles.

Placebos and blinding

People improve when they believe they are being treated, so a control group needs a placebo — an inert pill, a sham procedure — rather than nothing at all. Otherwise you measure the treatment plus the belief, and cannot separate them.

Single-blind means the subjects do not know which group they are in. Double-blind means the people measuring the outcome do not know either, since an expectant researcher rounds ambiguous readings in the hoped-for direction without any intention to cheat.

Double-blind, randomised, placebo-controlled is the standard because each word closes a specific hole.

When you cannot experiment

You cannot randomly assign people to smoke for thirty years, or to grow up in poverty. For such questions, observational studies are all there is, and the honest response is to strengthen them rather than pretend.

That is what was done for smoking: many studies, many populations, a dose-response pattern, a plausible mechanism, and no surviving confounder that explained it all. Causal language became reasonable — but it took decades of evidence to earn what one randomised experiment would have given immediately.

Common mix-ups

• Calling any large study an experiment. Size is not assignment.
• Confusing random sampling with random assignment. Sampling decides who is in the study, and governs whether results generalise. Assignment decides who gets the treatment, and governs whether you can claim cause. A study can have either, both, or neither.
• Thinking a control group alone is enough. Without randomisation the groups may have differed from the start.
• Reading a confounder as a measurement error. It is a real variable doing real work.
• Claiming cause from an observational study because the sample was big or the correlation strong.

Check yourself

Which kind of study can establish cause, what is a variable affecting both treatment and outcome called, and what is the name for assigning subjects to groups by chance?

Only a randomised experiment, because the researcher assigns the treatment. A variable tangled up with both is a confounding variable. And assigning by chance is randomisation.