Question

When an independent variable has more than three levels, why should you conduct an ANOVA and post-hoc testing instead of just multiple t-tests?

Answer 1

The t-test compares the means between 2 samples and is simple to conduct, but if there is more than 2 conditions in an experiment a ANOVA is required. The fact the ANOVA can test more than one treatment is a major advantage over other statistical analysis such as the t-test.T-tests are easier to conduct, so why not conduct a t-test for the possible interactions in the experiment? A Type I error is the answer because the more hypothesis tests you use the more you risk making a type I error and the less power a test has.

Once an Analysis of Variance (ANOVA) test has
been completed, the researcher may still need to
understand sub group differences among the dif-
ferent experimental and control groups. The sub-
group differences are called “pairwise” differences.
ANOVA does not provide tests of pairwise diffe-
rences. When the researcher needs to test pairwi-
se differences, follow-up tests called post hoctests
are required.
ANOVA output does not provide any analysis of
pairwise differences, so how shall the researcher
in vestigate differences among the various sub-
groups tested with ANOVA? The first approach
that comes to mind is to perform a number of
t-tests between each of the pairs of interest. This is
not a good approach for two reasons: First, doing
repeated statistical tests on the same data – which
is what performing t-tests on each pair of in te rest
does – causes alpha inflation (1). Second, the results
will still be uninterpretable because individual
t-tests can examine only two groups at a time.using t-tests to examine pairwise diffe-
rences is likely to overestimate the size of the indi-
vidual t-tests. This means that the sum of t-values
from all the pairwise t-tes ts will often exceed the
value of the t-statistic produced by one of the mul-
tiple comparison analysis statistics (2). As a result,
performing multiple t-tests will lead the researcher
to a higher probability of making a Type I error