Question

In: Statistics and Probability

Why would the researcher want to test multiple groups multiple times. Explain. Share your example

Why would the researcher want to test multiple groups multiple times. Explain. Share your example

Solutions

Expert Solution

Some things show the behaviour over time and these processes are called time series processes. In time series a researcher collects data over a fixed time period basis and then tries to capture patterns in data.

Multiple comparisons:

In statistics, the multiple comparisons, multiplicity or multiple testing problem occurs when one considers a set of statistical inferences simultaneously or infers a subset of parameters selected based on the observed values. In certain fields it is known as the look-elsewhere effect.

The more inferences are made, the more likely erroneous inferences are to occur. Several statistical techniques have been developed to prevent this from happening, allowing significance levels for single and multiple comparisons to be directly compared. These techniques generally require a stricter significance threshold for individual comparisons, so as to compensate for the number of inferences being made.

Multiple comparisons arise when a statistical analysis involves multiple simultaneous statistical tests, each of which has a potential to produce a "discovery." A stated confidence level generally applies only to each test considered individually, but often it is desirable to have a confidence level for the whole family of simultaneous tests.[4] Failure to compensate for multiple comparisons can have important real-world consequences, as illustrated by the following examples:

  • Suppose the treatment is a new way of teaching writing to students, and the control is the standard way of teaching writing. Students in the two groups can be compared in terms of grammar, spelling, organization, content, and so on. As more attributes are compared, it becomes increasingly likely that the treatment and control groups will appear to differ on at least one attribute due to random sampling error alone.
  • Suppose we consider the efficacy of a drug in terms of the reduction of any one of a number of disease symptoms. As more symptoms are considered, it becomes increasingly likely that the drug will appear to be an improvement over existing drugs in terms of at least one symptom.

In both examples, as the number of comparisons increases, it becomes more likely that the groups being compared will appear to differ in terms of at least one attribute. Our confidence that a result will generalize to independent data should generally be weaker if it is observed as part of an analysis that involves multiple comparisons, rather than an analysis that involves only a single comparison.

For example, if one test is performed at the 5% level and the corresponding null hypothesis is true, there is only a 5% chance of incorrectly rejecting the null hypothesis. However, if 100 tests are conducted and all corresponding null hypotheses are true, the expected number of incorrect rejections (also known as false positives or Type I errors) is 5. If the tests are statistically independent from each other, the probability of at least one incorrect rejection is 99.4%.

The multiple comparisons problem also applies to confidence intervals. A single confidence interval with a 95% coverage probability level will contain the population parameter in 95% of experiments. However, if one considers 100 confidence intervals simultaneously, each with 95% coverage probability, the expected number of non-covering intervals is 5. If the intervals are statistically independent from each other, the probability that at least one interval does not contain the population parameter is 99.4%.

Techniques have been developed to prevent the inflation of false positive rates and non-coverage rates that occur with multiple statistical tests

Classification of multiple hypothesis tests :

The following table defines the possible outcomes when testing multiple null hypotheses. Suppose we have a number m of null hypotheses, denoted by: H1, H2, ..., Hm. Using a statistical test, we reject the null hypothesis if the test is declared significant. We do not reject the null hypothesis if the test is non-significant. Summing each type of outcome over all Hi yields the following random variables:

Null hypothesis is true (H0) Alternative hypothesis is true (HA) Total
Test is declared significant V S R
Test is declared non-significant U T {\displaystyle m-R}
Total {\displaystyle m_{0}} {\displaystyle m-m_{0}} m
  • m is the total number hypotheses tested
  • {\displaystyle m_{0}} is the number of true null hypotheses, an unknown parameter
  • {\displaystyle m-m_{0}} is the number of true alternative hypotheses
  • V is the number of false positives (Type I error) (also called "false discoveries")
  • S is the number of true positives (also called "true discoveries")
  • T is the number of false negatives (Type II error)
  • U is the number of true negatives
  • {\displaystyle R=V+S} is the number of rejected null hypotheses (also called "discoveries", either true or false)

In m hypothesis tests of which {\displaystyle m_{0}} are true null hypotheses, R is an observable random variable, and S, T, U, and V are unobservable random variables.


Related Solutions

what would a researcher use this t-test for? Provide an example.
what would a researcher use this t-test for? Provide an example.
4. Why would a researcher test a one-tailed vs. a two-tailed hypothesis? 5. Why would it...
4. Why would a researcher test a one-tailed vs. a two-tailed hypothesis? 5. Why would it be impossible for a researcher to test the following hypothesis using a t-test? H1: Female homicide offenders are more likely to be sentenced to maximum security prison than male homicide offenders. 6. If the null hypothesis in a t-test for two samples is not rejected, what conclusion can be drawn about the two means of the categorical variable? 7. Two groups of subjects participated...
Identify the Test that is required and explain why: A researcher uses heart rate as a...
Identify the Test that is required and explain why: A researcher uses heart rate as a measure of excitability in rats. He knows the average heart rate in rats and compares his sample of rats to that average. The sample has been bred with the hope that as a group the rats have a less excitable temperament.
Please share an example of the "acid test ratio" and explain whether or not you think...
Please share an example of the "acid test ratio" and explain whether or not you think this ratio would be helpful in evaluating Loon Company's cash situation. Jill Land, president of Loon. Co., is happy to report to the company's stockholders that the company increased its cash position by 20% from last year. Its average collection period decreased by 12 days. Jill knows some customers are unhappy about the new credit terms but believes that you cannot please everyone; that's...
1. Why would a researcher need to use a two-tailed test vs. a one-tailed test? 2.A...
1. Why would a researcher need to use a two-tailed test vs. a one-tailed test? 2.A scholar tests the following hypothesis:  Females have a greater number of delinquent peers than males.  In her test, she calculates a t value is -2.349.  Why would it be unnecessary to compare this test statistic to a critical t value?
What statistical test would you conduct for each example below? (a) You want to determine if...
What statistical test would you conduct for each example below? (a) You want to determine if there is an association between age and cholesterol (HDL). You have three age groups and you’ve grouped HDL into low/med/high. (b) You want to see if there’s an association between the number of days exercised in the past month (0–31 days) and blood pressure (SBP). (c) You’d like to compare mean patient satisfaction scores (0–100) across three different units in the hospital. (d) You’d...
A researcher conducts a hypothesis test and concludes that his hypothesis is correct. Explain why this...
A researcher conducts a hypothesis test and concludes that his hypothesis is correct. Explain why this conclusion is never an appropriate decision in hypothesis testing.
In your own words, explain why you would use a nonparametric test instead of a parametric...
In your own words, explain why you would use a nonparametric test instead of a parametric test. How are they different? How are they the same? Come up with a research study example for each of the different kinds of Chi square tests: no difference, goodness of fit, and test of independence.
Provide your own example of a situation where it would be appropriate to test for an...
Provide your own example of a situation where it would be appropriate to test for an interaction using a two-way independent ANOVA?
Would you want to know if your spouse or lover cheated on you? Why or why...
Would you want to know if your spouse or lover cheated on you? Why or why not ? Does being married make a difference in how you feel about this issue? Why or why not?
ADVERTISEMENT
ADVERTISEMENT
ADVERTISEMENT