Question

Define reliability & validity, describe and separate the three methods for demonstrating validity. talk about the relationship between validity and reliability

Answer 1

Reliability refers to the consistency of scores obtained on a test. Validity of a test refers to the accuracy with which the test measures what it claims to measure.

There are three methods for demonstrating validity:

1. Content Validity: Content validity is determined by the degree to which the questions, tasks, or items on a test are representative of the universe of behavior the test was designed to sample. In theory, content validity is really nothing more than a sampling issue. The items of a test can be visualized as a sample drawn from a larger population of potential items that define what the researcher really wishes to measure. If the sample (specific items on the test) is representative of the population (all possible items), then the test possesses content validity.

2. Criterion validity: Criterion-related validity is demonstrated when a test is shown to be effective in estimating an examinee’s performance on some outcome measure. In this context, the variable of primary interest is the outcome measure, called a criterion. The test score is useful only insofar as it provides a basis for accurate prediction of the criterion. For example, a college entrance examination that is reasonably accurate in predicting the subsequent grade point average of examinees would possess criterion-related validity. Two different approaches to validity evidence are subsumed under the heading of criterion-related validity. In concurrent validity, the criterion measures are obtained at approximately the same time as the test scores. For example, the current psychiatric diagnosis of patients would be an appropriate criterion measure to provide validation evidence for a paper-and-pencil psychodiagnostic test. In predictive validity, the criterion measures are obtained in the future, usually months or years after the test scores are obtained, as with the college grades predicted from an entrance examination.

3. Concurrent Validity: In a concurrent validation study, test scores and criterion information are obtained simultaneously. Concurrent evidence of test validity is usually desirable for achievement tests, tests used for licensing or certification, and diagnostic clinical tests. An evaluation of concurrent validity indicates the extent to which test scores accurately estimate an individual’s present position on the relevant criterion. For example, an arithmetic achievement test would possess concurrent validity if its scores could be used to predict, with reasonable accuracy, the current standing of students in a mathematics course. A personality inventory would possess concurrent validity if diagnostic classifications derived from it roughly matched the opinions of psychiatrists or clinical psychologists.

The relationship between validity and reliability is complex. A valid test will always be reliable. If the test accurately assesses a construct then the scores of the individual on that construct will be consistent. However, a reliable test may not always be valid. A test may be consistently assessing an attribute which may not be the same as what it claims to measure.