The Observations That Aren't There

The data in a spreadsheet can look reassuringly official. Rows, columns, units, tidy headings. Even the occasional blank cell seems manageable. Surely a computer can just ignore a few missing values.

It can. Whether it should is a much more interesting question.

A real dataset, not a textbook invention

R distributes a dataset named airquality, comprising daily environmental measurements in New York from May through September 1973.

The R documentation describes 153 daily records and six variables: ozone, solar radiation, wind, temperature, month, and day.

The ozone readings were recorded at Roosevelt Island; meteorological observations came from Central Park and LaGuardia Airport. The observations did not all come from a single instrument or location, an important detail before we interpret their relationships.

The dataset includes 37 missing ozone readings and seven missing solar-radiation readings. Temperature and wind are recorded for all 153 days.

These missing cells are not merely nuisance characters for a spreadsheet. They are part of the evidence we have.

Audit the data before analyzing it

In R, the first step is remarkably simple:

data("airquality", package = "datasets")
dim(airquality)
colSums(is.na(airquality))

This tells us the dataset’s shape and the missing-value count for every variable.

Now consider two different analyses:

mean(airquality$Temp)
mean(airquality$Ozone, na.rm = TRUE)

The temperature mean uses 153 observations. The ozone mean, after removing missing values, uses 116. They do not summarize the same number of days.

R’s mean() on a vector containing NA returns NA unless we specify how missingness should be handled. That’s inconvenient, but scientifically useful: it prevents a silent assumption.

The denominator changes inside regression, too

Suppose we fit a linear model describing ozone concentration in relation to solar radiation, wind, and temperature:

model <- lm(Ozone ~ Solar.R + Wind + Temp, data = airquality)
nrow(airquality)
nobs(model)

A complete-case regression uses only rows where all variables in the model are present. For this particular dataset and model, 111 observations remain.

So an analysis that began with 153 days uses only 111 for its regression. The other 42 rows did not contribute to the fitted coefficients. We should report that explicitly.

The total number of rows removed is not necessarily 37 + 7: some rows can be missing both ozone and solar radiation.

Three different missingness questions

Statistics distinguishes several mechanisms:

• Missing completely at random (MCAR): missingness is unrelated to observed or unobserved data under the model.
• Missing at random (MAR): after conditioning on appropriate observed information, missingness does not additionally depend on the unobserved values.
• Missing not at random (MNAR): missingness depends on unobserved information even after conditioning as specified.

The phrase missing at random is unfortunately named. It does not mean that rows disappeared through a magical coin toss. It names a particular conditional statistical assumption.

We usually cannot determine the true missingness mechanism merely by inspecting the observed dataset. Subject-matter expertise, data-collection documentation, and sensitivity analyses matter.

A complete-case analysis may be reasonable under some assumptions. Under others, it can produce biased estimates or poorly represent particular conditions or dates.

A reproducible audit

Open the R Workbench from this lesson. The accompanying script counts missing values, compares denominators, identifies the rows used by the regression, and plots observed temperatures against ozone for records where both are available.

Do not interpret the fitted regression as demonstrating that temperature or wind caused ozone readings. The variables are observational, measurements came from different locations and times, and seasonality or other variables could affect the relationship.

Exercise

1. How many nonmissing ozone observations appear in the 153-row dataset?
2. Does a regression with 111 complete cases necessarily tell us why the other 42 rows are missing?
3. If missing ozone readings disproportionately occurred on high-ozone days, what might happen to the observed-case average?

Suggested responses

1. $153-37=116$.

2. No. A complete-case model simply excludes those rows; understanding why data are absent requires additional investigation.

3. The observed-case average could underestimate the all-day mean. That conclusion depends on the mechanism, and the missing ozone values themselves are not observed.

Provenance and data access

Source: R Core’s official documentation for datasets::airquality, citing New York State Department of Conservation and National Weather Service data, and Chambers, Cleveland, Kleiner, and Tukey, Graphical Methods for Data Analysis (1983).

We use the dataset as distributed with R and do not redistribute a copy here. It is historical, not a description of current New York air quality.

A sensitivity analysis that changes the unseen values

Consider a synthetic survey of 100 selected people. Eighty respond, with 48 supporters; 20 do not respond. Respondent support is 60%. Without assumptions about nonrespondents, population support among the 100 selected people could range from $48/100=0.48$ if none of the missing people support the proposal to $68/100=0.68$ if all do. This is an identification bound, not a confidence interval: it describes what missing outcomes alone permit before sampling uncertainty.

A simple sensitivity parameter assigns nonrespondents support probability $q$. The full-group expected support becomes $(48+20q)/100=0.48+0.2q$. Matching respondent support sets $q=0.6$ and returns 0.60. Choosing $q=0.2$ gives 0.52; choosing 0.9 gives 0.66. The observed responses cannot choose among these assumptions by themselves.

Missing at random is conditional: after conditioning on the included observed information, missingness does not additionally depend on the unseen value under that model. It is not a claim that every person has an equal response probability. Multiple imputation generally propagates uncertainty across plausible completed datasets; filling every missing value with a single mean instead treats invented values as known and often understates uncertainty and distorts associations.

Further problems

1. A sample has 30 observed successes, 50 observed failures, and 20 unknown outcomes. Find worst-case bounds on its complete-group success fraction.
2. Why does complete-case regression change both sample size and potentially the population represented?
3. What should a reproducible missingness audit report before comparing two fitted models?

Worked further solutions

1. Lower bound 0.30, upper bound 0.50.
2. Inclusion requires every needed variable to be observed. Those retained may differ systematically from those omitted.
3. Missing counts by variable, overlapping patterns, the row inclusion rule, retained identifiers or a reproducible mask, and whether both models use the same observations.