The Data the Regression Line Missed

A line can be mathematically optimal and still be the wrong description of the relationship.

Ordinary least squares will happily fit a line to observations arranged in a parabola. It will find the line that minimizes the squared vertical distances. The algorithm has done exactly what we asked.

The problem is that we asked the wrong question.

Look at the mistakes, not just the answers

For each observation, the residual is

$$e_i=y_i-\widehat y_i.$$

A positive residual means the observed value lies above its fitted value. A negative residual means it lies below.

Plotting residuals against the predictor, or against fitted values, can reveal things that a single regression coefficient conceals.

If a linear model describes the mean relationship reasonably well, residuals should not trace an obvious curve. Their spread and unusual points still need examination. Absence of an obvious pattern, however, is not proof that all model assumptions hold.

What does R-squared mean?

For a least-squares regression with an intercept, we can calculate

$$R^2=1-\frac{\sum_i(y_i-\widehat y_i)^2}{\sum_i(y_i-\bar y)^2}.$$

The denominator is the total sum of squared deviations about the sample mean. The numerator is the remaining sum of squared residuals after fitting the line.

In this ordinary in-sample setting, $R^2$ summarizes the proportion of sample variation accounted for by the fitted model, on this sum-of-squares scale.

A value close to one means the fitted line accounts for much of the in-sample variability. It does not mean that the model is probably true, that a prediction will be equally accurate for new observations, or that $x$ causes $y$.

There are important edge cases. When every observed $y$ is identical, the total sum of squares is zero and the usual fraction does not define $R^2$.

The curve hidden inside the line

The laboratory generates nine synthetic observations and fits a line. At first, the data are nearly linear.

As you increase the curvature, the relationship bends. The least-squares line adjusts, but it cannot reproduce the curve. The residuals begin to show a systematic pattern: observations are systematically above the line in some regions and below it elsewhere.

That is a clear visual warning that the linear form may be inadequate.

A better model might include a quadratic term or another scientifically appropriate functional form. But adding more terms simply to maximize $R^2$ can overfit noise. Model choice should use substantive knowledge, diagnostics, and honest out-of-sample evaluation.

Correlation and nonlinear dependence

Pearson correlation can be close to zero even when two variables have a strong curved relationship. Similarly, a fitted linear slope might be close to zero in symmetric curved data.

The sensible response isn’t to throw away statistics. It is to use the correct statistical question.

Exercise

1. A line predicts 12, but the observed value is 17. What is the residual?
2. Why might a U-shaped residual plot cast doubt on a straight-line mean model?
3. Does $R^2=.9$ imply a 90 percent probability that the model is correct?

Worked responses

1. $17-12=5$.

2. A systematic U-shaped pattern indicates that the model systematically overpredicts or underpredicts across ranges of the predictor; it misses a curved relationship.

3. No. $R^2$ is a descriptive ratio of sums of squares for the fitted sample, not a posterior probability or a causal claim.

Read further

See OpenStax’s regression chapter. For research-oriented analysis, residual diagnostics should also consider unequal variance, dependence, influential observations, and model stability.

A diagnostic sequence with distinct questions

A residual-versus-fitted plot examines whether the mean shape is appropriate and whether residual spread changes with fitted level. A residual-versus-time plot checks dependence or drift hidden by an unordered scatterplot. A normal quantile plot compares residual shape with a normal reference, relevant to exact small-sample inference. None is an automatic pass/fail certificate, particularly with a small sample.

Leverage measures how unusual a predictor configuration is. In simple regression with an intercept, $h_{ii}=1/n+(x_i-\bar x)^2/S_{xx}$. For the five-point example with $\bar x=3$, $S_{xx}=10$, endpoints at 1 and 5 have leverage 0.6, while the center at 3 has leverage 0.2. A large residual and high leverage together can make one observation influential. A high-leverage observation can also have a small fitted residual because it pulls the line toward itself.

With an intercept, the identity $SS_T=SS_{reg}+SSE$ makes $R^2=1-SSE/SS_T$. Adding regressors cannot increase least-squares SSE on the same observed rows, so in-sample $R^2$ cannot decrease. This is not proof of better prediction. Compare held-out error, defensible model structure, and sensitivity, and keep the analysis sample fixed when attributing a change in fit to a predictor.

Further problems

1. If $SSE=20$ and $SS_T=50$, calculate $R^2$.
2. What residual pattern suggests increasing variance?
3. Why is automatic deletion of an influential observation not a diagnostic solution?

Worked further solutions

1. $1-20/50=0.6$.
2. A fan-shaped spread increasing with fitted values or predictor level, subject to context and sample limitations.
3. Influence may identify a legitimate important case, a model failure, or a recording error. Investigate provenance and report sensitivity; deleting it changes the evidence.