Question

“I don’t like statistical tests,” Ben says grumpily. “There’s so much chance for error, especially Type I error. I don’t understand why we don’t just set alpha really low, like 0.01 or even zero, and have a much lower chance of error.”

What would you say to Ben, in 200 words or less?

Answer 1

It is not about reducing the error. If you see clearly there are two types of errors currently present while we conduct a hypothesis testing. They are Type-I error and Type-II error. In an ideal situation, we would love to minimize both errors. But in very rare cases we can do such minimizing simultaneously. But we want to see which error is more severe. In general, we see that type-I error is more severe. So we would love to minimize type-I error on a given level of level significance. But if we set the sett the alpha very low that might lead to type-II error very low and type-I error to high, which is adequately not required in daily life problem. So first we see the required problem, we set the hypothesis, we set the null hypothesis and alternative hypothesis, we set the critical region according to our testing based on a given level of significance. by this process, we see that if we get a type-I error and type-II error and if we get type-I error<type-II error then we can conclude we can work on that level of significance. Thus by choosing a low-level alpha like 0.01 will not solve our problem which may or may not lead to lower type-I error.

Let us consider an example:

Let us consider a prisoner. Now there are two scenarios. The prisoner is guilty or he is not guilty. Now if he is guilty and the judge says that he is innocent then we consider it as a type-II error. Again if he is not guilty and the judge announces him not innocent and sentenced him to jail, that is a type-I error. So here we see that type-I error is more severe like sending an innocent person to jail. So here we see that we have to minimize the type-I error. Thus by this example, we see that we have to minimize the type-I error by setting the correct alpha.