Question

The concepts of reliability and validity are very important when it comes to the testing of intellectual abilities. When tests are given such importance (in terms of how we look at and use the numbers), they must be carefully constructed to be as accurate as possible. Could a test of intelligence be reliable, but not valid? How? Could some other measure (assessing something other than intelligence) be valid, but NOT reliable? Why or why not? Give examples.

Of what practical use are intelligence tests? Does it matter what someone's "IQ" is? Why or why not? Do you think that you could tell the difference between two people whose IQ scores differ by 10 points? By 20? (Assume normal, everyday type conversations.) At what size gap do you think there WOULD be a noticeable difference? What behavior(s) or other observations would signal to you that a person has a higher IQ score?

Answer 1

It is possible for a test to be highly reliable but not valid. Reliablity indicates consistency, that is, the degree to which a test produces consistent and stable results. Suppose we construct an intelligence test in which the items are very similar to each other but cover only a single domain of verbal intelligence, say, reading comprehension. The reliability of such a test would be very high because the items are similar to each other and therefore participants would be inclined to responding similarly to all the items. However, since the items would only be assessing a specific form of verbal intelligence, i.e., reading compression skills while being called an intelligence test, its construct validity would be very poor as it would be covering only a single domain of intelligence as a broad construct.

Similarly, there maybe cases wherein a test may have high validity but low reliability. Suppose the test was constructed in a manner in which all the items equally represented different aspects of intelligence such as numerical intelligence, verbal intelligence etc. and correlated well with existing intelligence tests. However, the individual items in the test differed drastically from each other and the inter-item correlation was very low. In such a case, the test could be called valid but not reliable. During test construction, therefore, researchers must carefully aim to achieve a balance between validity and reliability.

Please post the other questions separately.