Question

How do you determine if the data/information you are using for an analysis is reliable and valid? (You'll have to consider the definitions of reliability and validity.)
- How critical is this type of concern for the Health Services Area?

Answer 1

QUESTION:

How do you determine if the data/information you are using for an analysis is reliable and valid? (You'll have to consider the definitions of reliability and validity.)
- How critical is this type of concern for the Health Services Area?

ANSWER:

RELIABILITY AND VALIDITY

INTRODUCTION

Validity and reliability are two important factors to consider when developing and testing any instrument (e.g., content assessment test, questionnaire) for use in a study. Test reliability and validity are two technical properties of a test that indicate the quality and usefulness of the test.

VALIDITY

• Validity is the extent to which a test measures, what it is supposed to measure.
• The question of validity is raised in the context of the three points:

-the form of the test

-the purpose of the test

-the population for whom it is intended.

Types of Validity

• External validity
• Internal Validity
• Content Validity
• Face Validity
• Test Validity

a.Criterion Validity

b.Predictive Validity

c.Concurrent Validity

• Construct Validity

a.Convergent Validity

b.Discriminant validity

1. External Validity

• External validity occurs when the causal relationship discovered can be generalized to other people, time and contexts.
• Correct sampling will allow generalization and hence give external validity.

2. Internal validity:

• Internal validity occurs when it can be concluded that there is a causal relationship between the variables being studied.
• It is related to the design of the experiment.

3. Content Validity:

• When we want to find out if the entire content of the behavior/construct/area is represented in the test we compare the test task with the content of the behavior.
• This is a logical method, not an empirical one.
• Example, if we want to test knowledge on American Geography it is not fair to have most questions limited to the geography of New England.

4. Face Validity:

• Face validity occurs where something appears to be valid.
• This depends very much on the judgment of the observer.

5. Test Validity:

    a) Criterion: Correlation with the standards.

    b) Predictive: Predicts future values of criterion.

   c) Concurrent: Correlates with other test.

6. Construct Validity:

    Construct validity accurately represent reality.

a) Convergent: Simultaneous measure of same construct correlate.

b) Discriminant: Doesn’t measure what it shouldn’t.

RELIABILITY

• Reliability is the degree to which a test consistently measures whatever it measures.
• When a measurement procedure yields consistent scores when the phenomenon being measured is not changing
• Degree to which scores are free of “Measurement Error
• Consistency of the measurement

Types of reliability

1.Stability Reliability

     a.Test- retest

2.Equivalence Reliability

     a.Inter term -Parallel forms

     b.Inter observer Agreement

1. Stability Reliability:

a.Test-retest:

• Test-retest reliability is the degree to which scores are consistent over time.
• It indicates score variation that occurs from testing session to testing session as a result of errors of measurement.
• Same test- different Times
• Only works if phenomenon is unchanging
• Example: Administering the same questionnaire at 2 different times

2. Equivalence Reliability:

a) Inter-item reliability: (internal consistency)

• The association of answers to set of questions designed to measure the same concept
• Cronbach’s alpha is a statistic commonly used to measure inter-item reliability which is based on the average of all the possible correlations of all the split 1/2 s of set of questions on a questionnaire.

b) Parallel form of Reliability

Split-Half Reliability:

• Especially appropriate when the test is very long.
• The most commonly used method to split the test into two is using the odd-even strategy.
• Since longer tests tend to be more reliable,and since split-half reliability represents the reliability of a test only half as long as the actual test.

C) Inter observer Reliability:

• Correspondence between measures made by different observers.

Relationship between validity & reliability

• Necessary but not sufficient
• Reliability is prerequisite for measurement of validity
• One needs reliability but it is not enough

Importance of validity and reliability for the health services area

• The adoption of evidence-based medicine and evidence-based management, i.e. focusing on underlying organizational issues that influence how care is delivered, by clinicians, administrators, payers and regulators demands that healthcare marketers be knowledgeable in assessing and applying credible and applicable evidence Doing so, especially within the context of marketing or social science research, begins with an understanding of the concepts of validity and reliability.
• It is also important that validity and reliability not be viewed as independent qualities. A measurement cannot be valid unless it is reliable; it must be both valid and reliable if it is to be depended upon as an accurate representation of a concept or attribute .A research study design that meets standards for validity and reliability produces results that are both accurate (validity) and consistent (reliability). The archery metaphor is often used to illustrate the relationship between validity and reliability.
• Knowledge of validity and reliability not only aids the researcher in designing and judging one’s own work, it also makes one a better consumer of research through the ability to evaluate research literature and in choosing among alternative research designs and interventions .Adopting these standards will ensure that study results are credible to key constituents.
• Tt assure the quality of newly invented procedures.
• These measures enhance the qualities through which modifications or innovations are done in health care.

CONCLUSION

It is important to understand the differences between reliability and validity. Validity tells that  how good a test is for a particular situation; reliability tells that how trustworthy a score on that test will be.