Home / Lean Six Sigma / ANOVA, regression & simulation

ANOVA: comparing means with one or more factors

Paste your data from Excel and find out whether suppliers, machines, shifts or process settings give different means, which pairs differ, how much each factor weighs and whether the analysis assumptions hold. Use it when there are 3 or more groups (for just 2, the t-test is enough): paste one column with the group and one with the measurement, and the page shows which group is best and what it is worth per year.

Pre-sizing and estimating tool: results do not replace design, calculations or reports signed by a licensed professional, nor verification against the applicable codes and standards.

CheckRevised 11/03/2026
Data

Paste from Excel with a header row. Stacked: one column per factor and one for the response (up to 4 factors, unbalanced data accepted). Side by side: one column per group.

See what was read

Role of each column

Analysis options
Practical interpretation (optional)

Your inputs stay only on this browser and are not sent anywhere. Untick to delete them.

—ANOVA F-test
—

Analysis results

Method

Factor information

Analysis of variance

Welch’s test

Model summary

Means

Coefficients

Fits and diagnostics for unusual observations

R: large standardized residual (|R| > 2). X: unusual X (high leverage).

How to read: Analysis of variance: DF = degrees of freedom; SS (Seq/Adj) = variation attributed to each source; MS = SS ÷ DF; F = between-group variation ÷ within-group variation (the larger, the more the groups differ); P-value < α = the factor changes the mean. Model summary: S = remaining spread; R² = % of the variation explained by the factors.

How to read: in the grouping table, levels that do not share a letter have significantly different means; levels with a letter in common may be equal. In the differences table, each row is a pair: an interval that excludes zero or an adjusted P-value < α = a different pair.

Intervals for differences of means

Each row is a pair. If the interval does not cross zero, the difference is significant (in red).

How to read: each row is a pair of levels (e.g. B − A) and the bar is the interval for the difference between the means. The vertical line is zero: a bar that does not touch zero (red) = the two levels really differ; a bar that crosses zero = no evidence of a difference. The further from zero, the larger the difference.

Tap the chart to see the values.

Interval plot

Mean of each level with its confidence interval. The dashed line is the overall mean.

How to read: each dot is the mean of a level and the vertical bar the range where the true mean probably lies. Bars that do not overlap suggest different levels; long bars indicate few data or much spread in the group.

Tap the chart to see the values.

Means comparison chart

Comparison intervals: two levels whose intervals do not overlap differ. In red, those that do not overlap at least one other.

How to read: unlike the interval plot, here the bars are adjusted so the visual rule works: if two levels’ bars do not touch, the means differ; if they touch, there is no evidence. Use it to present the comparison results in an easy-to-see way.

Tap the chart to see the values.

Boxplot by level

Box = middle 50% (Q1 to Q3), line = median, × = mean, whiskers to 1.5 × IQR; isolated points are unusual data.

How to read: the vertical axis is the response and each box a level. Compare the height of the boxes (level) and their size (spread). ANOVA assumes similar spreads: one box much bigger than the others calls for the variance test and perhaps the Welch option. Isolated points should be checked.

Tap the chart to see the values.

Individual value plot

Each measurement by level, with the means connected across levels.

How to read: each dot is a real measurement and the line connects the level means. If the clouds of points barely overlap, the difference is visible by eye; if they mix a lot, the difference in means is small compared with part-to-part variation, even if it is significant.

Tap the chart to see the values.

Test for equal variances

Tests

How to read: each bar is the interval for one level’s standard deviation (Bonferroni-corrected for several comparisons). Bars at the same height that overlap = compatible spreads, classic ANOVA holds. A level with a much higher bar is less stable: investigate it and use Welch. A Levene P-value < α confirms different spreads.

Residual plots (four in one)

Residual = measurement − fitted mean of its group. They check the assumptions: normality, constant variance and independence.

Normal probability plot

Versus fits

Histogram

Versus order

How to read: the four panels check whether the ANOVA can be trusted. Normal probability: points on the line = normal residuals. Versus fits: a band of constant width around zero = equal variance; a funnel shape = not. Histogram: bell shape. Versus order: points with no trend or cycles = independent measurements; drift over time is a warning sign.

Next step

Take this page’s results to another calculator with the fields already filled in.

The calculation report gathers this page's inputs, results, equations, assumptions and references, with fields for project, engineer and revision. The full report also includes the charts and interpretations, in color. In the print dialog, choose Save as PDF.

Method, assumptions and references

Equations used

SS total = Σ(yᵢ − ȳ)²; SS factor = Σ nⱼ(ȳⱼ − ȳ)²; SS error = Σ(yᵢ − ȳⱼ)²; F = MS factor ÷ MS error, with MS = SS ÷ DF
S = √MS error; R² = 1 − SS error/SS total; R²(adj) = 1 − MS error/(SS total/(N − 1)); R²(pred) = 1 − PRESS/SS total, PRESS = Σ[eᵢ/(1 − hᵢ)]²
General linear model: y = Xβ + ε with factors coded (−1, 0, +1); sequential (type I) and adjusted (type III, dropping each term from the full model) sums of squares
Welch: F* = [Σwⱼ(ȳⱼ − ȳw)²/(k − 1)] ÷ [1 + 2(k − 2)/(k² − 1)·Σ(1 − wⱼ/W)²/(nⱼ − 1)], wⱼ = nⱼ/sⱼ²
Tukey-Kramer: ȳᵢ − ȳⱼ ± q(1 − α; k; DF error)/√2 · S·√(1/nᵢ + 1/nⱼ); Fisher: ± t(1 − α/2; DF)·SE; Dunnett: ± d·SE (multivariate t); Games-Howell: Welch SE and DF for each pair
Levene (Brown-Forsythe): ANOVA of |yᵢ − medianⱼ|; Bartlett: T = [(N − k)ln s²p − Σ(nⱼ − 1)ln sⱼ²]/C; Bonferroni CI for the standard deviation: s·√((n − 1)/χ²), with α/k

Assumptions and limits

  • Independent observations, collected in random order across levels; approximately normal residuals (with 15 or more per level, ANOVA tolerates moderate departures).
  • Fixed factors (the levels studied are the ones of interest). For random factors (variance components), use the gage R&R study or a mixed model.
  • Power in the report card uses the F-test with the pooled standard deviation (approximation); the reference Assistant uses Welch. Hsu MCB and Games-Howell only in the one-factor model.

Standards and technical documents

  • Montgomery, D. C. Design and Analysis of Experiments, 10th ed. Wiley, 2019 (cap. 3, 5 e 15)
  • Hsu, J. C. Multiple Comparisons: Theory and Methods. Chapman & Hall, 1996
  • Dunnett, C. W. A multiple comparison procedure for comparing several treatments with a control. JASA 50:1096–1121, 1955
  • Welch, B. L. On the comparison of several mean values: an alternative approach. Biometrika 38:330–336, 1951
  • Brown, M. B.; Forsythe, A. B. Robust tests for the equality of variances. JASA 69:364–367, 1974
  • Hochberg, Y.; Weiss, G.; Hart, S. On graphical procedures for multiple comparisons. JASA 77:767–772, 1982
  • ISO 3534-3:2013 Statistics — Vocabulary and symbols — Part 3: Design of experiments
  • NIST/SEMATECH e-Handbook of Statistical Methods, seções 7.4 e 5.3

U.S. regulations

  • ISO 9001:2015, 9.1.3: analysis and evaluation (including statistical techniques)
  • 21 CFR 820.250 (statistical techniques, Quality System Regulation for medical devices)

Frequently asked questions

When should I use ANOVA instead of a t-test?

A t-test compares two means. With three or more levels (for example, four suppliers), running several t-tests raises the chance of a false alarm: with six pairs at 5% each, the chance of at least one error exceeds 25%. ANOVA tests them all at once and Tukey comparisons control that error.

How do I read the P-value in the ANOVA table?

It is the probability of seeing differences as large as those observed if all means were actually equal. In the supplier example, p < 0.001: the difference is very unlikely to be chance. With p ≥ 0.05 you cannot claim a difference, which does not prove the means are equal.

Which comparison should I pick: Tukey, Fisher, Dunnett or Hsu?

Tukey compares all pairs and is the default choice. Dunnett compares each level with a control (the current supplier) and is more sensitive when that is all you need. Hsu MCB identifies which level may be the best. Fisher does not correct for many pairs and is only a screening tool.

What if the variances are different?

Choose "Assume equal variances: No". The calculator then uses Welch’s test and Games-Howell comparisons, which do not require equal spreads. Remember that a much more variable level is a quality finding in itself: it will produce more out-of-spec parts even with the right mean.

How many measurements do I need per level?

For ANOVA to be robust to non-normal data, 15 or more per level (20 with 10 to 12 levels), the Assistant criterion used in the report card. To detect a small difference, enter the difference that matters: the card shows the power and how many measurements 80% and 90% power would take.

What is an interaction and how do I see it in the plot?

There is an interaction when the effect of one factor changes with the level of another. In the roughness example, if a higher feed worsened the finish much more at one cutting speed than at another, the lines in the interaction plot would fan out or cross. Parallel lines mean the effects simply add up.

Why is the sequential SS different from the adjusted SS?

In unbalanced data (levels with different numbers of measurements), the factors are partly confounded. Sequential SS depends on the order in which terms enter; adjusted SS measures each term after all others and is the one used for the F-test. In balanced data both are the same.