Home / Lean Six Sigma / Statistical analysis

ANOVA: comparing means with one or more factors

Paste your data from Excel and find out whether suppliers, machines, shifts or process settings give different means, which pairs differ, how much each factor weighs and whether the analysis assumptions hold.

Pre-sizing and estimating tool: results do not replace design, calculations or reports signed by a licensed professional, nor verification against the applicable codes and standards.

CheckRevised 11/03/2026
Data

Paste from Excel with a header row. Stacked: one column per factor and one for the response (up to 4 factors, unbalanced data accepted). Side by side: one column per group.

See what was read

Role of each column

Analysis options
Practical interpretation (optional)

Your inputs stay only on this browser and are not sent anywhere. Untick to delete them.

—ANOVA F-test
—

Analysis results

Method

Factor information

Analysis of variance

Welch’s test

Model summary

Means

Coefficients

Fits and diagnostics for unusual observations

R: large standardized residual (|R| > 2). X: unusual X (high leverage).

Intervals for differences of means

Each row is a pair. If the interval does not cross zero, the difference is significant (in red).

Tap the chart to see the values.

Interval plot

Mean of each level with its confidence interval. The dashed line is the overall mean.

Tap the chart to see the values.

Means comparison chart

Comparison intervals: two levels whose intervals do not overlap differ. In red, those that do not overlap at least one other.

Tap the chart to see the values.

Boxplot by level

Box = middle 50% (Q1 to Q3), line = median, × = mean, whiskers to 1.5 × IQR; isolated points are unusual data.

Tap the chart to see the values.

Individual value plot

Each measurement by level, with the means connected across levels.

Tap the chart to see the values.

Test for equal variances

Tests

Residual plots (four in one)

Residual = measurement − fitted mean of its group. They check the assumptions: normality, constant variance and independence.

Normal probability plot

Versus fits

Histogram

Versus order

Next step

Take this page’s results to another calculator with the fields already filled in.

The calculation report gathers this page's inputs, results, equations, assumptions and references, with fields for project, engineer and revision. The full report also includes the charts and interpretations, in color. In the print dialog, choose Save as PDF.

Method, assumptions and references

Equations used

SS total = Σ(yᵢ − ȳ)²; SS factor = Σ nⱼ(ȳⱼ − ȳ)²; SS error = Σ(yᵢ − ȳⱼ)²; F = MS factor ÷ MS error, with MS = SS ÷ DF
S = √MS error; R² = 1 − SS error/SS total; R²(adj) = 1 − MS error/(SS total/(N − 1)); R²(pred) = 1 − PRESS/SS total, PRESS = Σ[eᵢ/(1 − hᵢ)]²
General linear model: y = Xβ + ε with factors coded (−1, 0, +1); sequential (type I) and adjusted (type III, dropping each term from the full model) sums of squares
Welch: F* = [Σwⱼ(ȳⱼ − ȳw)²/(k − 1)] ÷ [1 + 2(k − 2)/(k² − 1)·Σ(1 − wⱼ/W)²/(nⱼ − 1)], wⱼ = nⱼ/sⱼ²
Tukey-Kramer: ȳᵢ − ȳⱼ ± q(1 − α; k; DF error)/√2 · S·√(1/nᵢ + 1/nⱼ); Fisher: ± t(1 − α/2; DF)·SE; Dunnett: ± d·SE (multivariate t); Games-Howell: Welch SE and DF for each pair
Levene (Brown-Forsythe): ANOVA of |yᵢ − medianⱼ|; Bartlett: T = [(N − k)ln s²p − Σ(nⱼ − 1)ln sⱼ²]/C; Bonferroni CI for the standard deviation: s·√((n − 1)/χ²), with α/k

Assumptions and limits

  • Independent observations, collected in random order across levels; approximately normal residuals (with 15 or more per level, ANOVA tolerates moderate departures).
  • Fixed factors (the levels studied are the ones of interest). For random factors (variance components), use the gage R&R study or a mixed model.
  • Power in the report card uses the F-test with the pooled standard deviation (approximation); the reference Assistant uses Welch. Hsu MCB and Games-Howell only in the one-factor model.

Standards and technical documents

  • Montgomery, D. C. Design and Analysis of Experiments, 10th ed. Wiley, 2019 (cap. 3, 5 e 15)
  • Hsu, J. C. Multiple Comparisons: Theory and Methods. Chapman & Hall, 1996
  • Dunnett, C. W. A multiple comparison procedure for comparing several treatments with a control. JASA 50:1096–1121, 1955
  • Welch, B. L. On the comparison of several mean values: an alternative approach. Biometrika 38:330–336, 1951
  • Brown, M. B.; Forsythe, A. B. Robust tests for the equality of variances. JASA 69:364–367, 1974
  • Hochberg, Y.; Weiss, G.; Hart, S. On graphical procedures for multiple comparisons. JASA 77:767–772, 1982
  • ISO 3534-3:2013 Statistics — Vocabulary and symbols — Part 3: Design of experiments
  • NIST/SEMATECH e-Handbook of Statistical Methods, seções 7.4 e 5.3

U.S. regulations

  • ISO 9001:2015, 9.1.3: analysis and evaluation (including statistical techniques)
  • 21 CFR 820.250 (statistical techniques, Quality System Regulation for medical devices)

Frequently asked questions

When should I use ANOVA instead of a t-test?

A t-test compares two means. With three or more levels (for example, four suppliers), running several t-tests raises the chance of a false alarm: with six pairs at 5% each, the chance of at least one error exceeds 25%. ANOVA tests them all at once and Tukey comparisons control that error.

How do I read the P-value in the ANOVA table?

It is the probability of seeing differences as large as those observed if all means were actually equal. In the supplier example, p < 0.001: the difference is very unlikely to be chance. With p ≥ 0.05 you cannot claim a difference, which does not prove the means are equal.

Which comparison should I pick: Tukey, Fisher, Dunnett or Hsu?

Tukey compares all pairs and is the default choice. Dunnett compares each level with a control (the current supplier) and is more sensitive when that is all you need. Hsu MCB identifies which level may be the best. Fisher does not correct for many pairs and is only a screening tool.

What if the variances are different?

Choose "Assume equal variances: No". The calculator then uses Welch’s test and Games-Howell comparisons, which do not require equal spreads. Remember that a much more variable level is a quality finding in itself: it will produce more out-of-spec parts even with the right mean.

How many measurements do I need per level?

For ANOVA to be robust to non-normal data, 15 or more per level (20 with 10 to 12 levels), the Assistant criterion used in the report card. To detect a small difference, enter the difference that matters: the card shows the power and how many measurements 80% and 90% power would take.

What is an interaction and how do I see it in the plot?

There is an interaction when the effect of one factor changes with the level of another. In the roughness example, if a higher feed worsened the finish much more at one cutting speed than at another, the lines in the interaction plot would fan out or cross. Parallel lines mean the effects simply add up.

Why is the sequential SS different from the adjusted SS?

In unbalanced data (levels with different numbers of measurements), the factors are partly confounded. Sequential SS depends on the order in which terms enter; adjusted SS measures each term after all others and is the one used for the F-test. In balanced data both are the same.