Question

Why is Spearman rank order used for test re-test reliability?

Answer 1

Spearman's rank correlation coefficient, which can be used to analyze test-retest reliability, was used to assess test-retest correlations for 12 personality items and 17 pathophysiological symptom 素證 items. The correlation coefficients for these items ranged from 0.444 to 0.802 (Table 3).

Table 3. Test–retest Reliability of the SCAT Questionnaire

	Item	Spearman rho
Personality
	Q1	0.589***
	Q2	0.698***
	Q3	0.695***
	Q4	0.614***
	Q5	0.735***
	Q6	0.707***
	Q7	0.651***
	Q8	0.642***
	Q9	0.755***
	Q10	0.649***
	Q11	0.652***
	Q12	0.750***
Pathophysiological symptoms (素證)
	Q13	0.513***
	Q14	0.762***
	Q15	0.642***
	Q16	0.700***
	Q17	0.649***
	Q18	0.577***
	Q19	0.706***
	Q20	0.725***
	Q21	0.742***
	Q22	0.669***
	Q25	0.444***
	Q27	0.784***
	Q28	0.802***
	Q29	0.704***
	Q30	0.707***
	Q31	0.690***
	Q32	0.764***

***

p < 0.001.

4. Discussion

The SCAT is an integrated Sasang constitutional analysis system developed by Korea Institute of Oriental Medicine based on constitutional information for 4300 people who were diagnosed by experts. Data for an individual's face, voice, body shape, and questionnaire results are entered into a SCAT program, Sasang constitution-related variables are extracted, and probability values for the four Sasang constitutional types are analyzed and provided. The SCAT is characterized by the provision of detailed and quantified information with respect to face, voice, body shape, and questionnaire responses, which is needed when a Korean medicine doctor diagnoses an individual's Sasang constitutional type.16, 17

This study was conducted to evaluate the reliability of the questionnaire component of the SCAT. The internal consistency reliability and test-retest reliability of questionnaire items were analyzed. Cronbach's alpha was used to determine whether the questionnaire items measure the same construct, and Spearman's rank correlation coefficient (Spearman's rho) was used to confirm the stability of the questionnaire items over a 4-week period.

Cronbach's alpha coefficient was assessed for 12 personality items and 13 pathophysiological symptom 素證 items for which internal consistency could reliably be analyzed. Cronbach's alpha was 0.788 for personality, 0.511 for eating habits, 0.718 for digestion, 0.667 for heat- or cold-wise penchant, and 0.612 for water ingestion (Table 2). In general, Cronbach's alpha coefficients of at least 0.6 are thought to be indicative of good reliability.18Therefore, this questionnaire was confirmed to exhibit internal consistency for all items except for those related to eating habits. Given that better internal consistency reliability is typically observed as the number of examined items increases, it appears possible that the internal consistency reliability of eating habit items is relatively low because there were only two items on eating habits. In the future, we can devise ways to improve internal consistency by increasing the number of eating habit items.

Spearman's rank correlation coefficient was used to assess test-retest correlations for 12 personality items and pathophysiological symptom 素證 items, and the correlation coefficients for these items ranged from 0.444 to 0.802 (Table 3). In general, correlation coefficients between 0.4 and 0.7 and between 0.7 and 0.9 indicate moderate and strong correlations, respectively.19 Therefore, given that the time interval between the two tests was 4 weeks, it can be confirmed that the questionnaire answers were relatively stable.

The results of this study demonstrated the internal consistency of the SCAT questionnaire measuring personality and pathophysiological symptoms and showed that the responses to each item remained stable over a 4-week period. However, the validity of the questionnaire was not evaluated in this study. Therefore, the sensitivity and specificity of the questionnaire should be evaluated in a future validation study. In addition, the study subjects were limited to university students in their twenties who have prior knowledge of Korean medicine and Sasang constitutional medicine. As a result, the capacity to generalize from the results of this study is limited and there might be a risk of bias due to the predefined subject group. Thus, additional reliability studies should be conducted for other age groups without prior knowledge of Korean medicine and Sasang constitutional medicine.