Script · Not a recorded lecture · Week 3

The answer changes when the respondents do

Distinguish descriptive association from population inference; inspect selection and clustering.

Written teaching script: This is the narration and demonstration plan for an eventual recorded lecture. No audio or video is claimed to exist. Mathematical and editorial review will precede filming.

Opening

Imagine an election in which everyone has already decided how to vote, but the poll numbers shift dramatically. No one changed their mind. We just changed who answered the telephone.

Teaching narrative

Start with the target population. If sixty percent belong to one group and forty percent to another, the citywide average depends on those population weights. A survey with different response rates changes the mix of observed people, sometimes dramatically.

The selection-bias demonstration lets one group's response rate move while the actual support rates and population composition remain fixed. The apparent survey result changes. This isn't a standard-error issue and it cannot be fixed simply by gathering more of the same nonrepresentative respondents.

Stratification and clustering add further complications. Sampling within defined strata can improve representation and precision, while selecting only a few clusters can introduce correlated observations. The usual independent-sample standard error may be much too optimistic.

Finally, correlation belongs to description, not automatic causation. If two variables move together, ask whether a third variable, sampling process, or common cause could explain what you see.

Before the experiment

Can the sample estimate change from about 63% to 54% without changing the true population support of 54%?

Live demonstration

Use the survey-selection lab and move Group B's response probability from 30% to 90%, then back to zero.

Exit question

Name one selection problem that collecting ten times as many respondents would not automatically solve.

Assigned reading for this lecture

Complete week 3's applied assignment →

All fourteen lecture scripts →