The Chi-Square Test

The idea in one sentence

Chi-square adds up how far each observed count sits from the count you would have expected, in units that make “far” comparable across cells of different sizes.

The test for counts

Everything else in inference works on means and proportions — numbers you can average. Chi-square handles categorical data: how many people fell into each box.

Roll a die 60 times expecting 10 of each face, and get \(8, 12, 9, 15, 7, 9\). Is that a loaded die, or just what 60 rolls look like? Eyeballing the list cannot answer it, and neither can a mean, because “the average face rolled” is not a meaningful quantity.

The statistic

taken over every cell, where \(O\) is what you observed and \(E\) is what the null hypothesis expects.

Three design decisions are packed into that formula. Squaring stops overshoots and undershoots cancelling out. Dividing by \(E\) scales each gap against how big the cell is, so being 5 off in a cell expecting 10 counts far more than being 5 off in one expecting

1. And summing pools every cell’s disagreement into one number.

For the die: \(\frac{(8-10)^2}{10} + \frac{(12-10)^2}{10} + \dots = \frac{4+4+1+25+9+1}{10} = 4.4\)

A perfect match gives 0. The statistic is never negative, and bigger always means worse agreement.

Degrees of freedom

The statistic alone means nothing until you know how many cells were free to vary — six cells naturally disagree more than two, so the threshold must depend on the count.

For a goodness-of-fit test across \(k\) categories, \(df = k - 1\). The die has \(6 - 1 = 5\).

Why minus one? Because the counts must total 60. Fix any five and the sixth is forced, so only five of them were genuinely free.

For a two-way table of \(r\) rows and \(c\) columns,

A 3-by-4 table has \(2 \times 3 = 6\). Same reasoning: once the row and column totals are fixed, filling in a 2-by-3 block determines the rest.

Reading the result

Compare your statistic against the critical value for its degrees of freedom, or get the p-value from software. At \(df = 5\) and \(\alpha = 0.05\) the critical value is 11.07. The die scored 4.4, comfortably below, so there is no evidence of loading — which is not the same as evidence of fairness.

Large statistic, small p-value, reject the null. The direction is worth fixing in mind: chi-square is always a one-sided test in its tail, even though the underlying question has no direction.

Two tests, one formula

Goodness of fit asks whether one categorical variable matches a claimed distribution. Expected counts come from the claim: if the claim is a fair die, \(E = \frac{60}{6} = 10\).

Independence asks whether two categorical variables in a two-way table are related. Expected counts come from the totals:

which is just \(P(A)P(B)\) times \(n\) — exactly what independence would predict. Same formula afterwards, same table of critical values.

The condition that actually bites

Every expected count should be at least 5. The chi-square distribution is a continuous approximation to a discrete situation, and with tiny expected counts that approximation falls apart and inflates the statistic.

Note it is the expected counts that must clear 5, not the observed ones. An observed zero is fine. The usual fix is combining adjacent categories until the expected counts are large enough.

Common mix-ups

• Using percentages instead of counts. The formula needs raw frequencies; feeding it percentages silently changes the sample size to 100.
• Checking observed counts against the rule of 5 rather than expected ones.
• Forgetting \(df\) depends on the number of cells, not the sample size.
• Multiplying \(r \times c\) instead of \((r-1)(c-1)\).
• Reading a significant result as a cause. It says the variables are associated, and says nothing about why — that needs an experiment.

Check yourself

What does chi-square compare observed counts against, what are the degrees of freedom for a 3-by-4 table, and what kind of p-value does a large statistic give?

It compares observed counts with expected ones. A 3-by-4 table has \((3-1)(4-1) = 6\) degrees of freedom. And a large statistic sits far out in the tail, so the p-value is small.