Question

When would researchers need high internal validity? When would they need high external validity?

research methods pys230

Answer 1

Why researchers require high internal validity

Internal validity refers to how well an experiment is done, especially whether it avoids confounding (more than one possible independent variable [cause] acting at the same time). The less chance for confounding in a study, the higher its internal validity is.

Therefore, internal validity refers to how well a piece of research allows you to choose among alternate explanations of something. A research study with high internal validity lets you choose one explanation over another with a lot of confidence, because it avoids (many possible) confounds.

Inferences are said to possess internal validity if a causal relationship between two variables is properly demonstrated.A valid causal inference may be made when three criteria are satisfied:

• the "cause" precedes the "effect" in time (temporal precedence),
• the "cause" and the "effect" tend to occur together (covariation), and
• there are no plausible alternative explanations for the observed covariation (nonspuriousness)

In scientific experimental settings, researchers often change the state of one variable (the independent variable) to see what effect it has on a second variable (the dependent variable).For example, a researcher might manipulate the dosage of a particular drug between different groups of people to see what effect it has on health. In this example, the researcher wants to make a causal inference, namely, that different doses of the drug may be held responsible for observed changes or differences. When the researcher may confidently attribute the observed changes or differences in the dependent variable to the independent variable (that is, when the researcher observes an association between these variables and can rule out other explanations or rival hypotheses), then the causal inference is said to be internally valid.

In many cases, however, the size of effects found in the dependent variable may not just depend on

• variations in the independent variable,
• the power of the instruments and statistical procedures used to measure and detect the effects, and
• the choice of statistical methods (see: Statistical conclusion validity).

Rather, a number of variables or circumstances uncontrolled for (or uncontrollable) may lead to additional or alternative explanations (a) for the effects found and/or (b) for the magnitude of the effects found. Internal validity, therefore, is more a matter of degree than of either-or, and that is exactly why research designs other than true experiments may also yield results with a high degree of internal validity.

In order to allow for inferences with a high degree of internal validity, precautions may be taken during the design of the study. As a rule of thumb, conclusions based on direct manipulation of the independent variable allow for greater internal validity than conclusions based on an association observed without manipulation.

When considering only Internal Validity, highly controlled true experimental designs (i.e. with random selection, random assignment to either the control or experimental groups, reliable instruments, reliable manipulation processes, and safeguards against confounding factors) may be the "gold standard" of scientific research. However, the very methods used to increase internal validity may also limit the generalizability or external validity of the findings. For example, studying the behavior of animals in a zoo may make it easier to draw valid causal inferences within that context, but these inferences may not generalize to the behavior of animals in the wild. In general, a typical experiment in a laboratory, studying a particular process, may leave out many variables that normally strongly affect that process in nature.

Why researchers require high external validity

External validity is the validity of applying the conclusions of a scientific study outside the context of that study.In other words, it is the extent to which the results of a study can be generalized to other situations and to other people.In contrast, internal validity is the validity of conclusions drawn within the context of a particular study. Because general conclusions are almost always a goal in research, external validity is an important property of any study. Mathematical analysis of external validity concerns a determination of whether generalization across heterogeneous populations is feasible, and devising statistical and computational methods that produce valid generalizations.

Over 40 years ago, Campbell and Stanley published their seminal work on experimental and quasi-experimental designs for research, in which they raised issues about threats to internal validity (whether or not observed covariation should be interpreted as a causal relationship) that exist when researchers are not able to randomly assign participants to treatments.

It has been frequently argued that internal validity is the priority for research.However, in an applied discipline, the purpose of which includes working to improve the health of the public, it is also important that external validity be emphasized and strengthened.For example, it is important to know not only that a program is effective, but that it is likely to be effective in other settings and with other populations.

In an influential 1985 article, “Efficacy and Effectiveness Trials (and Other Phases of Research) in the Development of Health Promotion Programs,” Flay proposes a model that emphasizes internal and external validity at different stages of the research process and that would lead to the translation of research to practice.8 The two main research levels were “efficacy trials” and “effectiveness trials.” Efficacy trials were to be highly controlled studies that answered the question of whether a proposed intervention would have the desired effects under ideal circumstances. Effectiveness trials were to follow efficacy trials and were to be studies that carried out the proposed intervention in less controlled and more real-life situations. The argument was that a given public health intervention should be successful in both types of trials before it was ready for dissemination to and by public health practitioners.

Efficacy trials were to have high internal validity, and effectiveness trials were to have high external validity. Efficacy trials were more likely to be controlled experiments, such as randomized control trials of public health interventions, that have the virtue of high internal validity but often have the liability of low external validity(i.e., the groups, settings, or contexts in which findings would apply). It is axiomatic in social science research that there is an inverse relationship between internal and external validity. A key to internal validity is good measurement and study design, and representative sampling is necessary for inference.However, it may be useful to distinguish between inference derived from sample design and our ability to generalize, which is more dependent on judgment.

Historically, researchers have tended to focus on maximizing internal validity, with the idea that it is more important to know if a given public health intervention works under highly controlled conditions than it is to know if it will work among different population groups, organizations, or settings. Similarly, funding organizations and journals have tended to be more concerned with the scientific rigor of intervention studies than with the generalizability of results. The consequence of this emphasis on internal validity has been a lack of attention to and information about external validity, which has contributed to our failure to translate research into public health practice.

For instance, in the area of cancer prevention and control, there is a documented substantial lag between discovery and delivery of effective interventions. Recognition of this lag has been noted for at least 30 years, since the first National Cancer Institute–convened cancer control working groups issued reports in the 1970s. More recently, Balas and Boren found that it takes about 17 years to turn 14% of original research to the benefit of patient care.Similarly, the National Research Council concluded that, even when effective interventions have been developed, there often is a gap between scientific knowledge and clinical practice.In addition, minorities and underserved communities usually gain access to effective interventions more slowly than do other populations.

Thus, the idea that research would progress from efficacy trials to effectiveness trials to widespread dissemination has not become a reality for a number of reasons, not the least of which is the time and cost involved in this stepwise progress of research to practice.As a result of the failure of this model, practitioners are often unable to determine if a given study’s findings apply to their local setting, population staffing, or resources.Reviews indicate that reporting on external validity is provided far less often than is reporting on other methodological issues.However, there are several reasons for the lack of information on external validity being an important contributor to the failure to translate research into public health practice. Policy and administrative decision-makers are unable to determine the generalizability or breadth of applicability of research findings. Finally, systematic reviews and meta-analyses are limited in the conclusions that can be drawn when external validity data are not reported.