Correlation and linear regression

What you’ll learn

• Calculate and interpret product moment correlation.
• Use the regression of y on x for prediction.
• Distinguish interpolation from extrapolation and association from causation.

Understand the idea

A scatter diagram shows the form and strength of association and can reveal an outlier before any calculation. The product moment correlation coefficient $r$ measures linear association and lies between $-1$ and $1$. A value near zero need not rule out a strong curved relationship.

The least-squares regression of $y$ on $x$ predicts $y$ from a specified $x$ and minimises the squared vertical residuals. Its line passes through $(\bar x,\bar y)$. The regression of $x$ on $y$ is a different fit, so do not simply rearrange a regression equation to predict the other variable.

A prediction within the observed range is interpolation. Extrapolation requires the relationship to continue outside the available evidence and can be unreliable. Correlation alone does not establish cause: a third variable, selection effects or chance may explain the association.

Key formulae

$$S_{xy}=\sum xy-\frac{(\sum x)(\sum y)}{n}$$

$$r=\frac{S_{xy}}{\sqrt{S_{xx}S_{yy}}}$$

$$\hat y=a+bx,\qquad b=\frac{S_{xy}}{S_{xx}}$$

$$a=\bar y-b\bar x$$

Worked example

A dataset has $\bar x=4$, $\bar y=9$, $S_{xx}=20$, $S_{yy}=80$ and $S_{xy}=30$. Find the regression of $y$ on $x$ and the correlation coefficient.

1. The gradient is $b=30/20=1.5$.
2. The intercept is $a=9-1.5(4)=3$.
3. The correlation is $r$ $=30/\sqrt{20\times80}$ $=30/40$ $=0.75$.

Answer: $\hat y=3+1.5x$ and $r=0.75$.