Question

How does changing alpha change the results of a hypothesis test?

Answer 1

Alpha tells us the maximum allowable p value for which we can reject the null hypothesis.

Sometimes, change in alpha level results in rejection of null hypothesis and sometimes, it results in failure to reject the null hypothesis depending upon the p value and alpha level

(A) CASE 1 (null hypothesis rejection to failure to reject null hypothesis)

when p value is 0.025. At 0.05 alpha level, we will reject the null hypothesis as p value is less than alpha level. If we change the alpha level to 0.01, then we fail to reject the null hypothesis as p value is greater than alpha level

(B) CASE 2 (failure to reject null hypothesis to null hypothesis rejection )

when p value is 0.025. At 0.01 alpha level, we will not reject the null hypothesis as p value is more than alpha level. If we change the alpha level to 0.10, then we will reject the null hypothesis as p value is less than alpha level