Question

What does statistical significance mean? How do you know if something is statistically significant? What is the difference between statistical significance and practical significance?

Answer 1

H0: Also known as the null hypothesis, it is a statement of "no effect"
or "no difference" used in tests of significance. The test of
significance is used to obtain substantial evidence against H0.
? Ha
: Also known as the alternative hypothesis, is what we hope to be
correct in rejection of the null hypothesis.
? P-Value: A P-value is essentially the probability that the test statistic
you compute would acquire a value as extreme or more than what you
actually observe. This is after making the assumption that H0 is true.
A common rule is the smaller the P-value, the stronger the evidence is
against H0.
? What does the P-value have to do with significance level?
We can compare the P-value calculated with a fixed value. This fixed
value is called the significance level.
? Significance Level: Is denoted by the Greek letter alpha (?). If the P-
value we calculate is as small or smaller than ?, we say the data is
statistically significant at level ?

What does “statistically significant” mean?
? When we use the term "significant", we are not automatically
implying that the data is "important". It is simply a term that is used to
assess whether the evidence against the null hypothesis has reached
the standard set by ? only.
? For example, significance level at 0.05 is often expressed by the
statement: "The results were significant at (P< 0.05)", where P stands
for the P-value.
? The P-value is helpful in providing basic information rather than a
statement of significance, because we can then assess significance at
any level we choose.

Statistical versus Practical Significance
What is STATISTICAL SIGNIFICANCE?
? Statistical significance refers to the unlikelihood that mean differences
observed in the sample have occurred due to sampling error. Given a
large enough sample, despite seemingly insignificant population
differences, one might still find statistical significance.
What is PRACTICAL SIGNIFICANCE?
? Practical significance looks at whether the difference is large enough
to be of value in a practical sense.