Question

What does it mean to say that a particular result is statistically significant at the .05 level or at the .01 level? Is a result that is statistically significant at the .05 level automatically also significant at the .01 level? What about the reverse? Please explain your reasoning.

Answer 1

If a particular result is statistically significant at the .05 level, then this means that there is 95% chances that the alternate hypothesis or claim is true and only 5% chances of finding it wrong.

Similarly, If a particular result is statistically significant at the .01 level, then this means that there is 99% chances that the alternate hypothesis or claim is true and only 1% chances of finding it wrong.

If a result is statistically significant at 0.05 level, this means that the p value corresponding to hypothesis testing must be less than 0.05 level. Suppose, we find a p value = 0.03, this is statistically significant at 0.05 level of significance because it is less than 0.05, but it is not significant at 0.01 level because it is not greater than 0.01. So, statistically significant at 0.05 level does not means that the test will be significant at 0.01 under all cases.

If a result is statistically significant at 0.01 level, this means that the p value corresponding to hypothesis testing must be less than 0.01 level. Suppose, we find a p value = 0.003, this is statistically significant at 0.01 level of significance because it is less than 0.01 and it is also significant at 0.05 level because it is less than 0.05. So, statistically significant at 0.01 level, will also be significant at 0.05 under all cases.