Question

Statistical significance is found in a study, but the effect in reality is very small (i.e., there was a very minor difference in attitude between men and women). Were the results meaningful? An independent samples t test was conducted to determine whether differences exist between men and women on cultural competency scores. The samples consisted of 663 women and 650 men taken from a convenience sample of public, private, and non-profit organizations. Each participant was administered an instrument that measured his or her current levels of cultural competency. The cultural competency score ranges from 0 to 10, with higher scores indicating higher levels of cultural competency. The descriptive statistics indicate women have higher levels of cultural competency (M = 9.2, SD = 3.2) than men (M = 8.9, SD = 2.1). The results were significant t (1311) = 2.0, p <.05, indicating that women are more culturally competent than are men. These results tell us that gender-specific interventions targeted toward men may assist in bolstering cultural competency. Critically evaluate the scenarios you selected based upon the following points: Critically evaluate the sample size. Critically evaluate the statements for meaningfulness. Critically evaluate the statements for statistical significance.

Answer 1

Sample Sizes

Sample sizes used were 663 for women and 650 for men. The data was collected from three points that is public, private and non-profit organizations. Though the sample seems good enough, the entire population figure is not provided. Another problem is that sample sizes from the three locations were not disclosed which probably means that the samples were all added up and used in the analysis. Sample sizes from each location should be carefully determined based on the population of each location. Convenience sampling is error bound, the samples shouldn’t be summed up and used in the analysis. Each location should be examined separately. This is to allow for examination of the error and comparison of results. What the researchers have done in this study regarding the samples and their sizes is not statistically sound and that will affect the findings.

Meaningfulness and Statistical Significance Combined

Convenience sampling is a statistical model of obtaining representative samples by selecting the samples because of the ease of access. For this reason, convenient samples are mostly biased and bound to have errors with high probabilities. Thus, we cannot generalize our findings to the entire population and also make inference about the entire population. Since the sample is not representative of the population, the results of the study cannot speak for the entire population. If this is done, it will result in a low external validity of the study.

In this scenario, results from a convenience sample have been generalized and used to make conclusions about the entire population. This is a statistical effect.

“The results were significant t (1311) = 2.0, p <.05, indicating……………………….”

When we give significance, the calculated P-value must be provided. In this scenario an indefinite P-value of <0.05 has been quoted. This leads the interested parties in the dark since they do not know if the P-value was 0.049 or 0.01 and thus aren’t able to judge how strong or weak the significance was.

This is a grave error in statistics