Question

So, I am conducting a research project and I have many categorical variables that I am trying to run crosstab and chi-square tests on. The only issue is that I only recieved 81 respondents so a lot of my questions have less than 5 people who selected one answer over another. I am unsure what to do. Do I state in my discussion section that due to there being a lack of respondents that I could not test for significance on many of my variables? Or, is there another test that I can run that I am not thinking of?

Answer 1

First of all, I suggest you if possible to use some extra resources (like time and money) to increase the number of respondents. Anyway more data is good for getting more reliable outputs from any of tests applicable.

Second thing is the target population of respondents. If you make survey on same kind of persons(interms of age or profession) then most of them would have almost same answers for many such questions. So instead go for heterogeneous population and adopt some other kind of sampling methods like stratified sampling, so that the chances of getting responses for many questions will increase. Though this procedure need relatively more time.

One more thing I want to put here is, say there is one question and having many categorical options. So the respondent can either choose one or many options as a response under that question. But if you observe that some of the options in that particular questions are neither opted at all by any person or opted by only few as compared to other options, then it is better to not include such options in questionnaire. So only relevant and less number of options for each question would always help for analysis. So if you already collected data and don't want to modify the questionnaire, then merge such option(having less frequency) with other one which is somewhat related and consider the new option with frequency as sum of frequencies of merged options.

In chi-square test, there is no assumption that the observed frequency (number of respondents in your case) in each cell of a contingency table should greater than or equal to 5. Instead there is assumption that the expected frequency (row_sum*column_sum/total) in each cell should be greater than or equal to 5. So firstly check if this assumption is true for your data. Because expected frequency is calculated by using the sum of observed frequencies across rows and columns, so in most of the cases if observed values are less, then expected are also, but not always.

If the assumption of Chi-square test is not valid, then the alternative test available is Fisher's exact test.

In any statistical software like SPSS, R one can apply these tests on data.