Question

A study testing two different production processes produced statically significant results. The research team, however, also said that the results were not practically significant to the company. Explain.

Answer 1

As we don't knpw about what kind of production we are talking about. Let's stick to topic Statistical signficance vs Practical Significance

Statistical significance refers to whether the observed effect is larger than we would expect by chance, i.e. can we reject the null hypothesis that there is no effect. This is what is typically addressed by p-values associated with T-tests or ANOVAs etc.

Practical significance is about whether we should care/whether the effect is useful in an applied context. An effect could be statistically significant, but that doesn't in itself mean that it's a good idea to spend money/time/resources into pursuing it in the real world. The truth is that in most situations , the null hypothesis is never true. Two groups will almost never be *exactly* the same if you were to test thousands or millions of people. That doesn't mean that every difference is interesting.This is usually associated with effect size measures (e.g. Cohen's d; which has criteria for 'small', 'medium' and 'large' effects), but generally will also need to take into account the context of the particular study (e.g. clinical research will have different expectations than personality psychology in terms of what kind of effects can be expected).