Question

In research people are often worried about a Type I or II error. Imagine a scenario where a researcher cannot collect much data. Which error will this directly impact? What is one change the researcher can make to increase the power of their test? What is something they cannot change that impacts power?

Answer 1

We need to remember that small sample size will not increase the Type I error rate. You can decrease your risk of committing a type II error by ensuring your test has enough power. You can do this by ensuring your sample size is large enough to detect a practical difference when one truly exists. Hence a smaller sample of the data will directly impact Type II error.

the power of a hypothesis test is affected by three factors.

1) Sample size: the greater the sample size, the greater the power of the test.

2) Significance level:  The lower the significance level, the lower the power of the test.

3) The "true" value of the parameter being tested: The greater the difference between the "true" value of a parameter and the value specified in the null hypothesis, the greater the power of the test. That is, the greater the effect size, the greater the power of the test.

Thus to increase the power of the test the researcher may increase the sample size and significance level can be 0.05 or 0.10.

They cannot change the true value of the parameter being tested.