Question

In applying t-tests to your data and you find that your data are very non-normal. How would you expect that non-normality to affect your results if corrected by family-wise error rate (FWER)? How about by false discovery rate (FDR)?

Answer 1

If the sample size in 2 samples gets large, then the t-test is valid as the type 1 error rate(FWER) is controlled at the given significance level(usually 5%) even when the random variable, X doesn't follow the normal distribution. It's because of the fact that t-test is based on the mean values of two samples and . And thus, by the Central Limit Theorem(CLT), the distribution of these data, in repeated sampling, converges to a normal distribution, regardless of the distribution of X in the population. Also the estimator that t-test uses for the standard error of the sample means is consistent regardless of the distribution of X and so it is unaffected by non-normality.

The t-test only assumes that the means of the different groups are normally distributed but it doesn't assume that the population is normally distributed.

By the central limit theorem, we have that the means of samples from a population with finite variance approach follows a normal distribution irrespective of the distribution of the population. Sample means are normally distributed as long as the sample size is at least 20(some say 30). For a t-test to be valid on a smaller sample size, the population distribution would have to be normal (or approximately normal).

However, the t-test is invalid for small samples from non-normal distributions as the type error rate deviates from the given significance level (5%) and false discovery rate(FDR: proportion of falsely rejecting the null hypothesis when in fact it is true out of total rejections) may increase. But the t-test is valid for large samples from non-normal distributions due to CLT.