Question

Explain the concept of Type M error by describing how the results would have had to turn out in order for the trial to reach statistical significance(p<0.05).

Answer 1

Type M error relate to the probability that claims with confidence are far in magnitude from underlying effect sizes. Effect size estimates based on preliminary data(either within the study or elsewhere) are likely to be misleading because they are generally based on small samples, and when the preliminary results appears interesting, they are most likely biased toward unrealistically large effects.

Type m error is also called exaggeration ratio i.e., expectation of the absolute value of the estimate divided by the effect size, if statistically significant different from zero.

Problems with the exaggeration ratio arises when power is less than 0.5.

It is quite possible for a result to be significant at 5% level of significance- with 95% confidence interval that entirely excludes zero- and for there to be high chance, sometimes 40% or more, that this interval is on the wrong side and direction of zero.