Question

A researcher conducts an Analysis of Variance (ANOVA) test looking at the effect of different treatment program options (therapy, education and reentry services) and finds that drug use levels vary significantly among between the participants in each of these three groups. For example, the therapy group had lowest amount of drug use (mean=2.0), the education group had the highest (mean=7.0) and the reentry services group was in the middle (mean=6.0). The researcher follows the steps to conduct an ANOVA appropropriately. The researcher's math is correct and she correctly rejects the null. However, she goes on to make the additional conclusion/interpretation: "Therefore, the ANOVA shows that there are statistically significant differences between the therapy group and the reentry services group."

Please evaluate the additional conclusions the researcher makes: a. Is it correct or incorrect to make this additional conclusion using the ANOVA test? Why or why not ? [Note the question is not at all concerned about differences between bivariate and multivariate analysis techniques. It only pertains to understanding ANOVA.]

Answer 1

No! It is not appropriate to draw an additional conclusion based on the ANOVA result. At first, let me explain what the ANOVA is?

Analysis of Variance (ANOVA) is a statistical method, commonly used in all those situations where a comparison is to be made between more than two population means like the yield of the crop from multiple seed varieties. It is a vital tool of analysis for the researcher that enables him to conduct tests simultaneously. When we use ANOVA, it is assumed that the sample is drawn from the normally distributed population and the population variance is equal.

When you use ANOVA to test the equality of at least three group means, statistically significant results indicate that not all of the group means are equal. However, ANOVA results do not identify which particular differences between pairs of means are significant. Use post hoc tests to explore differences between multiple groups means while controlling the experiment-wise error rate.

So, it is not correct to draw such an additional pair-wise conclusion. You can use Bonferroni Procedure, Fisher’s Least Significant Difference (LSD) or Tukey’s Test to conduct pair-wise test.