In: Statistics and Probability
Statistical significance refers to the unlikelihood that mean differences observed in the sample have occurred due to sampling error. Given a large enough sample, despite seemingly insignificant population differences, one might still find statistical significance.Practical significance looks at whether the difference is large enough to be of value in a practical sense.
Statistical significance refers to whether the observed effect is larger than we would expect by chance, i.e. can we reject the null hypothesis that there is no effect. This is what is typically addressed by p-values associated with the various statistical testing problems, ANOVA and so on. On the other hand, practical significance is about whether we should care/whether the effect is useful in an applied context. An effect could be statistically significant, but that doesn't in itself mean that it's a good idea to spend money/time/resources into pursuing it in the real world. The truth is that in most situations , the null hypothesis is never true. Two groups will almost never be *exactly* the same if you were to test thousands or millions of people. That doesn't mean that every difference is interesting.This is usually associated with effect size measures (e.g. Cohen's d), but generally will also need to take into account the context of the particular study.
STATISTICALLY SIGNIFICANT BUT NOT CLINICALLY SIGNIFICANT:
Consider the following scenario:
First I took a group of 2,000 adults between 20-30 years old, all of whom suffer from constant tiredness. Then the participants were randomly divided into 2 groups, with 1000 participants in each. One group of participants (the intervention group) were given the new drug: energyboost. The other groups of participants (the control group) were given a dummy (placebo) pill. Nobody knew – neither the participants nor the researchers involved in the experiment – whether they were taking energyboost or the placebo. The participants took the pills for 4 weeks, 4 per day. We used a scale to measure participants’ levels of tiredness before and after the trial. This rated fatigue on a scale of 1 to 20; with 1 meaning the participant felt entirely well-rested and 20 meaning the participant felt entirely fatigued. The results revealed that:Â 90% of the participants in the energyboost group improved by 2 points on the scale. 80% of participants in the placebo group improved by 1 point on the scale. This difference between the groups was statistically significant (p < 0.05) meaning that, at the end of the 4 weeks, participants in the intervention group were significantly less tired than those in the control group.
In the intervention group, 90% of the participants improved by 2 points on the tiredness scale whereas 80% of the participants in the placebo group improved by 1 point on the tiredness scale. Is the difference between both groups remarkable? Would you buy my product to have a slightly higher probability of achieving 1 point less on a tiredness scale, compared with taking nothing? Perhaps not. You might only be willing to take this new pill if it were to lead to a bigger, more noticeable benefit for you. For such a small improvement, it might not be worth the cost of the pill. So although the results may be statistically significant, they may not be clinically important or clinically significant.
CLINICALLY SIGNIFICANT BUT NOT STATISTICALLY SIGNIFICANT:
A clinically important change in pain in shoulder pain patients varies between patients with intact rotator cuffs and those with a ruptured rotator cuff. Patients with acute pain or higher levels of pain intensity may require less change in pain than chronic pain patients for their changes to be considered clinically important. Though it may not be statistically significant for the entire population, but may be clinically significant for some. So although the results may be clinically significant, they may not be statistically significant.
Hopefully this will help you. If you are satisfied with the answer, give a like. In case of any query, do comment. Thanks