Week 10 · Suggested 75 minutes · First-pass draft
The danger of a beautiful small p-value
Design a study based on a meaningful effect, then evaluate what happens when many hypotheses are tested.
Applied assignment, not a stored grade. Instructions and model checks are provided for self-study or instructor adaptation. Work is not transmitted to this site.
What you will do
- Compute the probability of at least one false rejection in twenty independent tests at alpha .05 when all nulls are true.
- Compute the Bonferroni per-test threshold for twenty comparisons.
- Write a hypothetical study-power statement specifying effect, variance, sample size, and alpha assumptions.
- Explain how exploratory analysis can be reported honestly rather than hidden or prohibited.
What to submit
Two calculations and a one-page interpretation of power, selective reporting, and practical importance.
Self-check before finishing
Familywise false rejection is approximately 64.2% in the independent-null model; Bonferroni threshold .0025.
Assigned reading
Balance study-design risks and recognize selective reporting.
Instructor discussion guidance
Assess whether students distinguish a pre-specified claim from a result selected among many analyses.
Print a predict–experiment–explain worksheet → · Open browser R →