ANOVA Test (Analysis of Variance) Explained with Examples

Quick answer: Learn what ANOVA is, when to use one-way vs two-way ANOVA, how to interpret the F-statistic and p-value, and how it differs from a t-test.

What is ANOVA?

ANOVA, short for Analysis of Variance, is a statistical test used to compare the means of three or more groups at once, to determine whether at least one group differs significantly from the others.

Why Not Just Run Multiple T-Tests?

Running a separate t-test for every pair of groups inflates the overall chance of a false positive, since each individual test carries its own error rate that compounds across multiple comparisons. ANOVA tests all groups together in a single test, controlling this overall error rate properly.

One-Way ANOVA

Compares group means based on a single categorical factor, such as comparing exam scores across three different teaching methods.

from scipy import stats

group_a = [78, 82, 85, 90, 76]
group_b = [65, 70, 72, 68, 74]
group_c = [88, 91, 85, 93, 89]

f_stat, p_value = stats.f_oneway(group_a, group_b, group_c)
print(f"F-statistic: {f_stat:.2f}, p-value: {p_value:.4f}")

Interpreting the F-Statistic and P-Value

The F-statistic compares variation between groups to variation within groups. A large F-statistic suggests the groups differ more than would be expected by chance alone. A p-value below the chosen significance level, commonly 0.05, indicates the difference between groups is statistically significant.

ANOVA Tells You THAT Groups Differ, Not WHICH Ones

A significant ANOVA result only tells you that at least one group differs from the others somewhere. It does not identify which specific groups differ. A post-hoc test, such as Tukey's HSD, is needed afterward to pinpoint exactly which pairs of groups differ significantly.

from statsmodels.stats.multicomp import pairwise_tukeyhsd
import numpy as np

data = np.concatenate([group_a, group_b, group_c])
labels = ['A'] * len(group_a) + ['B'] * len(group_b) + ['C'] * len(group_c)

result = pairwise_tukeyhsd(data, labels)
print(result)

Two-Way ANOVA

Tests the effect of two independent categorical factors at once, such as teaching method and student gender, and also tests whether the two factors interact with each other.

Assumptions of ANOVA

• Observations are independent of each other

• Data in each group is approximately normally distributed

• Groups have roughly equal variance, known as homogeneity of variance

When these assumptions are meaningfully violated, a non-parametric alternative like the Kruskal-Wallis test is more appropriate.

ANOVA vs T-Test

Common Interview Questions

Why use ANOVA instead of running multiple t-tests?

Multiple t-tests inflate the overall false positive rate across the multiple comparisons. ANOVA tests all groups together in a single test, controlling this error rate correctly.

Does a significant ANOVA result tell you which specific groups differ?

No. It only confirms that at least one group differs somewhere. A post-hoc test like Tukey's HSD is needed to identify which specific pairs differ.