Script · Not a recorded lecture · Week 8

Two groups are not always two independent samples

Analyze independent means, paired changes, and differences in proportions.

Written teaching script: This is the narration and demonstration plan for an eventual recorded lecture. No audio or video is claimed to exist. Mathematical and editorial review will precede filming.

Opening

A before-and-after study measures five people twice. That's ten numbers, but it is not ten independent people. This is exactly where pressing the wrong button on the statistics software produces an impressive-looking mistake.

Teaching narrative

Independent groups each contribute their own sample mean and estimated variance. Welch's t procedure accommodates different sample variances without insisting that the two populations have identical spreads.

Paired measurements instead create one difference per pair. The paired t procedure is a one-sample t analysis of those differences, which can remove irrelevant person-to-person variation when the matching is real.

Binary outcomes lead to proportions. A risk difference is an arithmetic difference in probabilities, a risk ratio compares probabilities multiplicatively, and an odds ratio compares odds. They are not synonyms.

Finally, none of these procedures repairs a flawed comparison. A tiny p-value does not magically turn an observational group difference into an identified causal treatment effect.

Before the experiment

Why can treating paired scores as independent inflate or alter estimated uncertainty?

Live demonstration

Compare the two independent fictional score vectors in R with Welch's test, then compare the five before/after pairs, then calculate a risk difference.

Exit question

Describe a study that should use a paired procedure, and one that should not.

Assigned reading for this lecture

Complete week 8's applied assignment →

All fourteen lecture scripts →