Question

Why is Tukey's method more powerful than Bonferroni's method?

Answer 1

The Bonferroni procedure is a good all around tool, but for all pairwise comparisons the Tukey studentized range procedure is slightly better as we show here.

The studentized range is the distribution of the difference between the maximum and a minimum over the standard error of the mean. When we calculate a t-test, or when we're using the Bonferroni adjustment where g is the number of comparisons, we are not comparing apples and oranges. In one case (Tukey) the statistic has a denominator with the standard error of a single mean and in the other case (t-test) with the standard error of the difference between means as seen in the equation for t and q above.

Here is an example we can work out. Let's say we have 5 means, so a = 5, we will let α = 0.05, and the total number of observations N = 35, so each group has seven observations and df = 30.

If we look at the studentized range distribution for 5, 30 degrees of freedom, (the distribution can be found in Appendix VII, p. 630.), we find a critical value of 4.11.

If we took a Bonferroni approach - we would use g = 5 × 4 / 2 = 10 pairwise comparisons since a = 5.  Thus, again for an α = 0.05 test all we need to look at is the t-distribution for α / 2g = 0.0025 and N - a =30 df . Looking at the t-table (found in Appendix II, p. 614) we get the value 3.03. However, to compare with the Tukey Studentized Range statistic, we need to multiply the tabled critical value by √2=1.4142=1.414, therefore 3.03 x1.414 = 4.28, which is slightly larger than the 4.11 obtained for the Tukey table.

The point that we want to make is that the Bonferroni procedure is slightly more conservative than the Tukey result, since the Tukey procedure is exact in this situation whereas Bonferroni only approximate.

Tukey is less conservative, that's why power is more