Question

Briefly describe the different post hoc tests (e.g., LSD, HSD, Fisher’s protected t tests, Bonferroni test) and why they are useful.

Answer 1

Post Hoc Test-

Post-hoc means to analyze the results of experimental data. They are often based on a family wise error rate; the probability of at least one Type I error in a set (family) of comparisons.

Some types of Post Hoc tests are:

• Bonferroni Procedure
• Holm-Bonferroni Method
• Dunn’s Multiple Comparison Test
• Fisher’s Least Significant Difference (LSD)
• Holm-Bonferroni Procedure
• Newman-Keuls
• Rodger’s Method
• Benjamin-Hochberg (BH) procedure

i. Bonferroni Procedure -
This multiple-comparison post-hoc correction is used when we are performing many independent or dependent statistical tests at the same time. The problem with running many simultaneous tests is that the probability of a significant result increases with each test run.

ii. Holm-Bonferroni Method-
The ordinary Bonferroni method is sometimes viewed as too conservative. Holm’s sequential Bonferroni post-hoc test is a less strict correction for multiple comparisons.

iii. Duncan’s new multiple range test (MRT)-
When we run Analysis of Variance (ANOVA), the results will tell us if there is a difference in means. However, it won’t pinpoint the pairs of means that are different. Duncan’s Multiple Range Test will identify the pairs of means (from at least three) that differ. The MRT is similar to the LSD, but instead of a t-value, a Q Value is used.

iv. Fisher’s Least Significant Difference (LSD)-
A tool to identify which pairs of means are statistically different. Essentially the same as Duncan’s MRT, but with t-values instead of Q values.

v. Newman-Keuls -
Like Tukey’s, this post-hoc test identifies sample means that are different from each other. Newman-Keuls uses different critical values for comparing pairs of means. Therefore, it is more likely to find significant differences.

vi. Rodger’s Method -
Considered by some to be the most powerful post-hoc test for detecting differences among groups. This test protects against loss of statistical power as the degrees of freedom increase.

vii. Benjamin-Hochberg (BH) procedure-
If we perform a very large amount of tests, one or more of the tests will have a significant result purely by chance alone. This post-hoc test accounts for that false discovery rate.