Choosing statistical tests and evaluating conclusions

What you’ll learn

• Match a test to categorical counts, group means or paired-variable association.
• Calculate a test statistic using supplied assumptions and critical values.
• Interpret significance without claiming proof or causation.

Understand the idea

Choose the analysis from the question and data structure before looking for a favourable result. A chi-squared goodness-of-fit test compares observed category counts with counts expected under a stated model; observations must be independent and expected counts sufficiently large for the approximation. A Student's t-test can compare two independent population means when its distribution and variance assumptions are justified. Paired before-and-after data require an analysis of the pairs, not automatic treatment as independent groups.

The independent-samples Student's t formula shown here uses a pooled variance, assuming approximately normal populations with a common variance. State sample sizes and use the relevant degrees of freedom, $n_1+n_2-2$, for the supplied critical value. Do not substitute standard errors where the formula expects standard deviations. Error-bar overlap by itself is not a universal significance test, particularly when the type of error bar is unspecified.

Correlation asks whether two measured variables are associated. Spearman's rank coefficient assesses a monotonic association; the simple displayed formula applies without tied ranks. A positive value means larger values of one variable tend to accompany larger values of the other. Even a statistically significant correlation can arise with confounding factors and does not establish causation. Use the supplied critical value or probability with the stated sample size and test direction.

A null hypothesis states the no-difference or no-association model relevant to the test. A result significant at the 5% level means the chosen test would produce a result at least this extreme with probability at most 0.05 under that null model and its assumptions. It does not mean there is a 95% probability that the research hypothesis is true. Failure to reject a null is also not proof that treatments are identical; sample size, variability and practical effect size matter.

Key formulae

$$\chi^2=\sum\frac{(O-E)^2}{E}$$

$$s_p^2=\frac{(n_1-1)s_1^2+(n_2-1)s_2^2}{n_1+n_2-2}$$

$$t=\frac{|\overline{x}_1-\overline{x}_2|}{s_p\sqrt{1/n_1+1/n_2}}$$

$$r_s=1-\frac{6\sum d_i^2}{n(n^2-1)}\quad\text{(no tied ranks)}$$

Worked example

For six independent field sites, soil-moisture ranks are $1,2,3,4,5,6$ and plant-cover ranks are $1,2,3,4,6,5$, listed in matching site order. There are no ties. Calculate Spearman's coefficient. The supplied two-tailed 5% critical magnitude is $0.886$; reject the no-association null when $|r_s|$ exceeds it.

1. The rank differences are $0,0,0,0,-1,1$, so $\sum d^2=2$.
2. Use $r_s$ $=1-6(2)/(6(6^2-1))$ $=1-12/210$ $=0.942857\ldots$.
3. Since $0.943>0.886$, reject the no-association null at the stated level. The result supports positive association, not proof that soil moisture causes cover to increase.

Answer: $r_s\approx0.943$; a significant positive rank association under the stated test.