Question

(a) What is the relationship of the p-value and hypothesis testing?
(b) What is a p-value threshold of 0.05 mean? (in words)
(c) Why do you think that a p-value of 0.05 is used so often as a threshold? What is a situation when that value would be too large (i.e. a value much lower than 0.05 should be used)?

Answer 1

(a)

A p value is used in hypothesis testing to help us do not reject or reject the null hypothesis. The p value is the evidence against a null hypothesis. The smaller the p-value, the stronger the evidence that we reject the null hypothesis.

(b)

A p-value less than 0.05 (p value ≤ 0.05) is statistically significant. It indicates strong evidence against the null hypothesis, as there is less than a 5% probability the null is correct . Therefore, we reject the null hypothesis, and accept the alternative hypothesis.

(c)

In the majority of analyses, an alpha of 0.05 is used as the cutoff for significance. If the p-value is less than 0.05, we reject the null hypothesis that there's no difference between the means and conclude that a significant difference does exist.

Below 0.05, significant.

Over 0.05, not significant

Reducing the alpha level from 0.05 to 0.01 reduces the chance of a false positive (called a Type I error). At the 0. 01 level, less room is left for errors or mistakes because the test is more rigorous. The p-value is a measure of how much evidence we have against the null hypothesis. A p-value less than 0.01 will under normal circumstances mean that there is substantial evidence against the null hypothesis.