Question

How can we improve the results of the project when we fail to reject the null hypothesis?

Answer 1

The hypothesis to be tested is called the null hypothesis and given the symbol H0. The null hypothesis states that there is no difference between a hypothesized population mean and a sample mean.

When we fail to reject a null hypothesis is evidence that the null hypothesis is true, but it might not be particularly a strong evidence, and it certainly doesn't prove the null hypothesis.

Absence of evidence is not evidence of absence.

When we run a hypothesis test, we are looking for evidence that the null hypothesis is not true. If we don't find it, then that is certainly evidence that the null hypothesis is true, but how strong is that evidence? To know that, we have to know how likely it is that evidence that would have made you reject the null hypothesis could have eluded your search. That is, what is the probability of a false negative on your test?

This is related to the power, , of the test (specifically, it is the complement, 1-.)

Now, the power of the test, and therefore the false negative rate, usually depends on the size of the effect we are looking for. Large effects are easier to detect than small ones. Therefore, there is no single for an experiment, and therefore no definitive answer to the question of how strong the evidence for the null hypothesis is. Put another way, there is always some effect size small enough that it's not ruled out by the experiment.

we can guard against this by not making inference solely about detecting difference. For example, we can combine inferences about difference and equivalence so that one is not favoring the burden of proof on evidence of effect versus evidence of absence of effect.