Unbiased estimation and standard error

What you’ll learn

• Use unbiased estimators for population mean and variance.
• Distinguish standard deviation from standard error.
• Describe the distribution of a sample mean.

Understand the idea

An estimator is a statistic used to estimate an unknown population parameter. It is unbiased when its expectation over repeated samples equals that parameter. This property does not mean every observed estimate is close to the truth; spread still matters. For independent identically distributed observations, the sample mean is an unbiased estimator of the population mean.

The unbiased sample variance divides the sum of squared deviations from the sample mean by $n-1$. A convenient computational form uses $\sum x_i^2-(\sum x_i)^2/n$. Dividing by $n$ instead gives the descriptive variance of the observed values and is not the same estimator.

The standard error of a sample mean is its sampling standard deviation, $\sigma/\sqrt n$. It measures uncertainty in the estimated mean, whereas the population standard deviation describes individual variation. For a Normal population the sample mean is exactly Normal; for a sufficiently large independent sample, the central limit theorem often provides an approximate Normal model.

Key formulae

$$\hat\mu=\bar X$$

$$S^2=\frac{\sum X_i^2-(\sum X_i)^2/n}{n-1}$$

$$\operatorname{SE}(\bar X)=\frac{\sigma}{\sqrt n}$$

$$\bar X\sim N\!\left(\mu,\frac{\sigma^2}{n}\right)\quad\text{for a Normal population}$$

Worked example

For a sample of five observations, $\sum x=30$ and $\sum x^2=190$. Find unbiased estimates of the population mean and variance.

1. The sample mean is $\bar x=30/5=6$.
2. The corrected sum of squares is $190-30^2/5=10$.
3. Divide by $n-1=4$ to obtain $s^2=10/4=2.5$.

Answer: Estimated mean $6$ and estimated variance $2.5$.