Question

I am performing a test re-test on a survey. Besides chi square, what other statistical tests can I use to determine content and criterion validity?

Answer 1

There are different validity and they can be determined by the following tests:

1) Convergent Validity (within larger Construct Validity) - refers to indicators / questionnaire items of a specific construct / variable are converging towards / loading a large portion of the variance that is in common (there are several ways to measure convergent validity, 1 of them is to check Average Variance Extracted (AVE) 0.5 (Hair et al., 1998)

2) Discriminant Validity (within larger Construct Validity) - refers to indicators / questionnaire items within a construct / variable are strongly associated with each other, but are distinctly from other constructs / variables' indicators / questionnaire items (to determine there is discriminant validity, square roots of AVEs should be higher than the correlations between constructs / variables)

3) Content Validity - refers to what extent the measurement is representing the content (this can be determined by panel of judges' opinions)

4) Face Validity - can be determined by basic walk through the questionnaire items whether they are intended to measure a concept

5) Concurrent Validity (within larger Criterion Validity) - refers to the questionnaire's capability that can differentiate respondents who are known to be different e.g. Theory-X vs Theory-Y workers (to determine concurrent validity, check the scores from different groups of respondents in which they should scored differently)

6) Predictive Validity (within larger Criterion Validity) - refers to the questionnaire's capability to differentiate respondents with reference to future criterion e.g. interview's aptitude test predicting future job performance (this can be determined by by checking the scores of good vs poor performers which should be different)

7) External Validity - refers to what extent a research findings through its questionnaire can generalize to other sample groups (this can be determined by comparing the statistical results of 2 different sampled groups)

Reliability can be determined by the following tests:

1) Cronbach Alpha (for internal consistency test using equal loadings or average) - generally the Alpha's coefficient should be > 0.7

2) Composite Reliability (using actual loadings) - generally > 0.7 & higher values than respective Cronbach Alpha

3) Test-retest Reliability Method

4) Parallel Form Reliability Method

5) Inter-item Consistency Reliability Method

6) Split-half Reliability Method