Question

When conducting a statistical hypothesis test, a very small p value indicates strong evidence against the null hypothesis. Why?

Answer 1

• Under the “classical” approach, with a fixed predetermined value of , a test will produce a decision as to whether or not to reject H0.
• But merely comparing the observed test statistic with some critical value and concluding eg “using a 5% test, reject H0” or “reject H0 with significance level 5%” or “result significant at 5%” (all equivalent statements) does not provide the recipient of the results with clear detailed information on the strength of the evidence against H0.
• A more informative approach is to calculate and quote the probability value (pvalue) of the observed test statistic.
• This is the observed significance level of the test statistic – the probability, assuming H0 is true, of observing a test statistic at least as “extreme” (inconsistent with H0) as the value observed.
• The p-value is the lowest level at which H0 can be rejected.
• The smaller the p-value, the higher the significance because it tells the investigator that the hypothesis under consideration may not adequately explain the observation.