Question

Explain what hypothesis testing is and what its criticisms are.

Answer 1

Hypothesis testing is the use of statistics to determine the probability that a given hypothesis is true.

One of the main goals of statistical hypothesis testing is to estimate the P-value, which is the probability of obtaining the observed results, or something more extreme if the null hypothesis were true. If the observed results are unlikely under the null hypothesis, you reject the null hypothesis.

A fairly common criticism of the hypothesis-testing approach to statistics is that the null hypothesis will always be false if you have a big enough sample size. Therefore, since you know before doing the experiment that the null hypothesis is false, there's no point in testing it.

This criticism only applies to two-tailed tests, where the null hypothesis is "Things are exactly the same" and the alternative is "Things are different." A significant rejection of the null hypothesis in a two-tailed test would then be the equivalent of rejecting one of the two one-tailed null hypotheses.

A related criticism is that a significant rejection of a null hypothesis might not be biologically meaningful if the difference is too small to matter.