In: Statistics and Probability
Suppose I calculate the Cohen’s d for the difference between 2 means and find that it is very small—that is, the two means are very close together. Do I know if the 2 means are not significantly different?
Suppose, corresponding random variables be X and Y.
Test statistic is given by
Cohen's d is given by
In general,
Thus we observe that, value of test statistic is higher than that of Cohen's d (in general). So, small value of Cohen's d does not imply small value of test statistic. Thus value of test statistic may be significantly larger (than critical value) implying significant difference between two means.
Hence, we cannot conclude explicitly about difference of two means (whether significant or not) without knowing sample sizes (in particular, little difference between sample means in case of higher sample sizes produce the conclusion that obtained difference is statistically significant).