Question

A colleague of yours is faced with testing a relationship at the .05 level or significance or the .01 level of significance. What is your advice to your colleague? Why?

Answer 1

Solution:- We should use level of significance = 0.01

Reducing the alpha level from 0.05 to 0.01 reduces the chance of a false positive (called a Type I error) but it also makes it harder to detect differences or relationship with a t-test. Any significant results we get would be more reliable but there would probably be less of them.

The level of significance evidence is also determined using the below rule.

• Probability > 0.1: No evidence
• Probability between 0.05 and 0.1: Weak evidence
• Probability between 0.01 and 0.05: Evidence
• Probability between 0.001 and 0.01: Strong evidence
• Probability < 0.001: Very strong evidence

The alpha level you set relates to the formal decision you make rather than this informal interpretation, but both can be reported together.

Lower alpha levels are sometimes used when you are carrying out multiple tests at the same time. A common approach is to divide the alpha level by the number of tests being carried out. For example, if you needed to carry out 5 tests you might set your initial alpha level at 0.05 then divide it by 5 to obtain the alpha level of 0.01. Again, the reason for this is to reduce the Type I error risk.