Question

Discuss and define reliability; the methods to estimate reliability; the concept of error and discuss and define validity, and differentiate between content and criteria-related validation. Why are reliability and validity important to selection?

Answer 1

Reliability and Validity are two technical properties of a criterion predictor that provide the quality and usefulness of the predictor. These features are used to evaluate the suitability of a criterion predictor.

Reliability refers to the extent to which a criterion predictor produces consistent results if repeated measurements are made.

Reliable assessment tools produce dependable, repeatable and consistent information about people.

There are different types of reliability estimates and each is influenced by different sources of measurement error. The acceptable level of reliability will differ depending on the type of criterion predictor and the reliability estimate used.

The methods to estimate reliability are –

1) Test- retest reliability: Suppose that scores on a test are used as criterion predictor, then the test is administered to a group of people and then after a gap of two to four weeks the same test is given to the same group of people. Correlation analysis is used to determine the degree of correlation between the two sets of scores. The higher the correlation coefficient the greater is the reliability of the test.

2) Alternate or parallel form reliability: indicates how consistent the test scores are likely to be if a person takes two or more forms of test. Here a high correlation coefficient would mean that different forms of test are very similar and therefore does not make a difference which form of the test is taken by the person.

3) Inter-rater reliability - indicates how consistent test scores are likely to be if the test is scored by two or more raters.

4) Internal consistency reliability - indicates the extent to which items on a test measure the same thing.

The concept of error in reliability is reported by the standard error of measurement (SEM). It gives the margin of error that can be expected in an individual test score because of the imperfect reliability of the test. The SEM represents the degree of confidence that a person's "true" score lies within a particular range of scores.

Validity refers to how accurately a criterion predictor predicts the criterion of job success that is what characteristic would the test measure and how well the test measures the characteristic or success predictor.

Content validity: refers to whether the content of a selection procedure or selection instrument such as a test is representative of important aspects of performance on the job. Content validity is especially useful in those situations where the number of employees is not large enough to justify the use of empirical validation methods. To use content validity, an employer must determine the exact performance of a specific job and develop a selection procedure or selection instrument around an actual sample of the work that is to be performed.

Criterion-related validity: is established by collecting data and using correlation analysis to determine the relationship between a predictor and the criteria of job success. The degree of validity for a particular predictor is indicated by the magnitude of the coefficient of correlation (r), which can range from + 1 to – 1 represent perfect correlation. Zero represents a total lack of correlation or validity. A positive sign (+) on the coefficient of correlation means the two sets of data are moving in the same direction, whereas a negative (-) sign means the two sets of data are moving in opposite directions.

The difference between the two is that content related validity measures job related behaviors and criterion related validity measures the job performance based on test score.

Reliability and Validity are important indicators as they facilitate the decision making process of the HR personnel in the selection process. These help the decision maker to find out the parameters that distinguish successful performance from unsuccessful performance in the available job and to forecast a person’s future performance in that job.