Question

Assume you have several years of student-level longitudinal data on math test scores, demographic characteristics, and what school each student enrolls in each year.

a. What would be the problem with simply comparing the math test scores of students attending a traditional public school with the outcomes of students who attend a charter school, even controlling for student demographic characteristics? Would this comparison yield the causal effect of attending a charter school on math test scores?

b. Some charter schools are oversubscribed, and by law they are required to admit people by lottery. How would you use the lottery data to overcome selection problems? Would this method tell you how an average charter school affects math test scores?

Answer 1

a) The problem with just comparing the math scores of traditional v/s charter school students is that the students might be self-selecting in these schools based on their ability. For instance, High ability students may be going to traditional schools. Now since ability definitely affects the test scores, it creates an omitted variable bias as an ability which is not included in the model affects test scores and is correlated with one of the independent variables. Hence, the results will be biased and will not be causal.

b) Using the oversubscription in charter schools, we can use a randomized control trial (RCT) to get a causal impact. Since the lottery is random, we can create a treatment group of people who get admission and a control group of students who applied but did not get admission. Randomization ensures that there is no difference in ability on average between the treatment and control group. One can then compare the outcomes and find the effect of charter school enrollment on math test scores.