Question

Summarize the results from the famous RAND health care study and compare those results to current Medicaid studies.

Answer 1

Abstract: In 2008, a group of uninsured low-income adults in Oregon was selected by lottery to be given the chance to apply for Medicaid. This lottery provides a unique opportunity to gauge the effects of expanding access to public health insurance on the health care use, financial strain, and health of lowincome adults using a randomized controlled design. In the year after random assignment, the treatment group selected by the lottery was about 25 percentage points more likely to have insurance than the control group that was not selected. We find that in this first year, the treatment group had substantively and statistically significantly higher health care utilization (including primary and preventive care as well as hospitalizations), lower out-of-pocket medical expenditures and medical debt (including fewer bills sent to collection), and better self-reported physical and mental health than the control group.

1. INTRODUCTION In early 2008, Oregon opened a waiting list for a limited number of spots in its Medicaid program for low-income adults, which had previously been closed to new enrollment. The state drew names by lottery from the 90,000 people who signed up. This lottery presents the opportunity to study the effects of access to public insurance using the framework of a randomized controlled design Although there are literally hundreds of studies comparing the health and health care utilization of insured and uninsured populations (see Institute of Medicine (2003)), inferring the impact of health insurance from such comparisons is difficult because individuals with and without insurance coverage differ in many ways – such as income, employment, or health – that are likely to be correlated with the outcomes of interest.1 Random assignment of health insurance to some but not others would avoid such confounding, but has never been done before in the United States.2 In this paper we examine the effects of the Oregon Medicaid lottery after approximately one year of insurance coverage. We present comparisons of outcomes between the treatment group (those selected by the lottery who had an opportunity to apply for Medicaid) and the control group (those not selected and thus not able to apply for Medicaid). We also present estimates of the impact of insurance coverage, using the lottery as an instrument for insurance coverage. We organize our analysis around the potential costs and benefits of health insurance. On the cost side, we examine the impact of health insurance on increased health care utilization. On the benefit side, we examine the impact of health insurance not only on health, which is a standard topic of analysis, but also on consumption smoothing, which we proxy for with measures of financial strain. There is remarkably little empirical work on the impact of health insurance on consumption smoothing or financial strain, even though risk spreading is arguably the primary purpose of health insurance (see e.g. Zeckhauser 1970). A priori, the sign – let alone magnitude – of the impact of health insurance on these various outcomes is not obvious. Although the price and income effects of subsidized health insurance should increase health care utilization, it is possible that increasing primary care utilization or improving health could result in offsetting reductions in hospital or emergency department use. Similarly, although health insurance is expected to improve health through increases in the quantity and quality of health care, it is also possible that by reducing the adverse financial consequences of poor health, health insurance may discourage investments in health and thereby worsen health outcomes. Moreover, for our low-income study population, the value of Medicaid relative to the existing safety net options is uncertain. The impact of Medicaid could be attenuated (or potentially non-existent) if public health clinics and uncompensated care allow individuals to consume de facto free medical care similar to that of the insured. Medicaid’s impact would also be attenuated if – as is often conjectured – Medicaid itself is not particularly “good” insurance – at least in terms of being able to access health care providers. Our analysis draws on administrative data from hospital discharge, credit report, and mortality records, as well as on responses to a large mail survey that we conducted. The administrative data are objectively measured and should not be biased by the treatment and control groups differentially reporting outcomes, but they only cover a relatively narrow set of outcomes. The survey data allow examination of a much richer set of outcomes than is feasible with administrative data alone, but, with a 50 percent response rate, are subject to potential non-response bias. Our available evidence on this issue is reassuring, but limited; response rates and available pre-randomization measures appear reasonably balanced between treatment and control responders. Prior to looking at the data on outcomes for the treatment group, virtually all of the analysis presented here was pre-specified and publicly archived in a detailed analysis plan.4 This was designed to minimize issues of data and specification mining and to provide a record of the full set of planned analyses. Although pre-specification of hypotheses is the norm for randomized controlled medical trials, to our knowledge is it extremely rare in evaluation of social policy experiments.5 About one year after enrollment, we find that those selected by the lottery have substantial and statistically significantly higher health care utilization, lower out-of-pocket medical expenditures and medical debt, and better self-reported health than the control group that was not given the opportunity to apply for Medicaid. Being selected through the lottery is associated with a 25 percentage point increase in the probability of having insurance during our study period. This net increase in insurance appears to come entirely through a gross increase in Medicaid coverage, with little evidence of crowd-out of private insurance. Using lottery selection as an instrument for insurance coverage, we find that insurance coverage is associated with a 2.1 percentage point (30 percent) increase in the probability of having a hospital admission, an 8.8 percentage point (15 percent) increase in the probability of taking any prescription drugs, and a 21 percentage point (35 percent) increase in the probability of having an outpatient visit; we are unable to reject the null of no change in emergency room utilization, although the point estimates suggest that such use may have increased. In addition, insurance is associated with threetenths of a standard deviation increase in reported compliance with recommended preventive care such as mammograms and cholesterol monitoring. Insurance also results in decreased exposure to medical liabilities and out-of-pocket medical expenses, including a 6.4 percentage point (25 percent) decline in the probability of having an unpaid medical bill sent to a collection agency and a 20 percentage point (35 percent) decline in having any out-of-pocket medical expenditures. Since much medical debt is never paid, the financial incidence of expanded coverage thus appears to be both on the newly insured and on their medical providers. Finally, we find that insurance is associated with improvements across the board in our measures of self-reported physical and mental health, averaging two-tenths of a standard deviation improvement. These results appear to reflect improvements in mental health and also at least partly a general sense of improved well being; they may also reflect improvements in objective, physical health, but this is more difficult to determine with the data we now have available. Our estimates of the impact of public health insurance apply to able-bodied uninsured adults below 100 percent of poverty who express interest in insurance coverage. This is a population of considerable policy interest. In 2011, fewer than half of the states offered Medicaid coverage to able-bodied adults with income up to 100 percent of poverty absent specific categorical requirements (Kaiser 2011). As part of the 2010 Patient Protection and Affordable Care Act, starting in 2014 all states will be required to extend Medicaid eligibility to all adults up to 133 percent of the federal poverty level, with no financial penalties for many individuals in this income range who do not take up coverage (Kaiser 2010a, Kaiser 2010b, US GPO 2010). The rest of the paper is structured as follows. Section two provides background on the Oregon Medicaid program and the lottery design. Section three describes the primary data sources, and section four presents our empirical framework. Section five presents our main results. Section six discusses interpretation and extrapolation of our estimates. A number of appendices – referenced in the text – provide additional details. 2. OREGON’S MEDICAID LOTTERY The Oregon Health Plan (OHP) – created by one of the first federal waivers of traditional Medicaid rules – currently consists of two distinct programs: OHP Standard and OHP Plus.6 OHP Plus serves the categorically eligible Medicaid population, which includes (up to specific income thresholds) children and pregnant women, the disabled, and families enrolled in Temporary Assistance to Needy Families (TANF). OHP Standard, which is the program that was lotteried, is a Medicaid expansion program to cover low-income adults who are not categorically eligible for OHP Plus. Specifically, it covers adults ages 19 – 64, not otherwise eligible for Medicaid, who are Oregon residents, are U.S. citizens or legal immigrants, have been without health insurance for six months, have income below the federal poverty level (FPL), and have assets below $2,000. OHP Standard provides relatively comprehensive benefits with no consumer cost sharing. It covers physician services, prescription drugs, all major hospital benefits, mental health and chemical dependency services (including outpatient services), hospice care, and some durable medical equipment. Vision is not covered nor are non-emergency dental services. Wallace et al. (2008) estimate that, in 2001-2004, average annual Medicaid expenditures for an individual on OHP Standard were about $3,000. Most care is provided through managed care organizations. Monthly enrollee premiums range from $0 to $20 depending on income (with those below 10 percent of the FPL paying $0). At its peak in early 2002, about 110,000 people were enrolled in OHP Standard, about one-third the size of OHP Plus enrollment at that time. Due to budgetary shortfalls, OHP Standard was closed to new enrollment in 2004. By early 2008, attrition had reduced enrollment to about 19,000 and the state determined it had the budget to enroll an additional 10,000 adults. Therefore, in January 2008 the state reopened OHP Standard to new enrollment. Because the state (correctly) anticipated that the demand for the program among eligible individuals would far exceed the 10,000 available new enrollment slots, it applied for and received permission from the Centers for Medicare and Medicaid Services to add the new members through random lottery draws from a new reservation list.7 From January 28 to February 29, 2008, anyone could be added to the lottery list by telephone, by fax, in person sign-up, by mail, or online. The state conducted an extensive public awareness campaign about the lottery opportunity. To keep barriers to sign-up low, the sign-up form (shown in Appendix Figure A2) requested limited demographic information on the individual and any interested household member, and no attempt was made to verify the information or to screen for program eligibility at sign-up for the lottery. A total of 89,824 unique individuals were placed on the list during the five week window it was open. The state conducted eight lottery drawings from the list, roughly equally sized and evenly staggered from March through September 2008. Selected individuals won the opportunity – for themselves and any household member (whether listed or not) – to apply for OHP Standard coverage; treatment thus occurred at the household level. In total, 35,169 individuals – representing 29,664 unique households –- were selected by lottery. If individuals in a selected household submitted the appropriate paperwork within 45 days after the state mailed them an application and demonstrated that they met the eligibility requirements, they were enrolled in OHP Standard.8 About 30 percent of selected individuals successfully enrolled. There were two main sources of slippage: only about 60 percent of those selected sent back applications, and about half of those who sent back applications were deemed ineligible, primarily due to failure to meet the requirement of income in the last quarter corresponding to annual income below the poverty level, which in 2008 was $10,400 for a single person and $21,200 for a family of four (Allen et al. 2010). If they did successfully enroll in OHP Standard, individuals could remain enrolled indefinitely, provided that they re-certified their eligibility status every six months.

4. EMPIRICAL FRAMEWORK 4.1 Reduced form 4.1.1 Reduced form equation We estimate the reduced form effect of winning the lottery with the following equation: ihj LOTTERYh Xih Vih ihj y ? ? ? ? ? ? ? ? ? ? 0 1 2 3 (1) where i denotes an individual, h denotes a household and j ? J denotes a “domain” of related outcomes (such as health or financial strain). For example yij might be the self-reported health of individual i, which is one of the health measures in the health “domain” J. We define (sign) each outcome within a domain so that higher values all have the same interpretation within a domain (e.g. more health care use, more financial strain). As we discuss below, we will summarize the estimates within a domain by the standardized treatment effect across outcomes in that domain; we will also report estimates for individual outcomes and show p-values that are adjusted to account for the multiple outcomes examined within the domain. LOTTERY is an indicator variable for whether or not household h was selected by the lottery. The coefficient on LOTTERY (?1) is the main coefficient of interest, and gives the average difference in (adjusted) means between the treatment group (the lottery winners) and the control group (those not selected by the lottery). We denote by ?ih the set of covariates that are correlated with treatment probability (and potentially with the outcome) and therefore must be controlled for so that estimates of ?1 give an unbiased estimate of the relationship between winning the lottery and the outcome. In all our analyses, ?ih includes indicator variables for the number of individuals in the household listed on the lottery sign up form (hereafter “household size”); although the state randomly sampled from individuals on the list, as mentioned above the entire household of any selected individual was then considered selected and eligible to apply for insurance. As a result, selected (treatment) individuals are disproportionately drawn from larger household sizes.16 For outcomes in the survey data, ?ih also includes indicator variables for survey wave (and the interaction of these indicator variables with household size indicators) since the fraction of treatment individuals varies across the seven survey waves.17 We denote by Vih a second set of covariates that can be included to potentially improve power by accounting for chance differences in variables between treatment and control group but that are not needed for ?1 to give an unbiased estimate of the relationship between winning the lottery and the outcome. There are three potential sources of such variables: the “lottery list” demographic variables; prerandomization measures of outcomes in the credit report data and hospital discharge data; and the lottery draw to which the individual is assigned.18 Our analysis of survey data will not control for any Vih covariates; our analysis of administrative data will include lottery draw indicators as well as the prerandomization measure for the outcome analyzed in the hospital and credit report data. Appendix 2.3  shows that our results are not sensitive to other choices regarding the Vih covariates in either the survey or administrative data. In all of our analyses we cluster the standard errors on the household identifier since the treatment is at the household level. Analyses of survey data are weighted to account for the sampling design of the survey as described above (and in Appendix 1.7). 4.1.2 Handling many outcomes: standardized treatment effects and multiple inference19 We summarize multiple findings across related outcomes within a domain J by the average standardized treatment effect: ? j?J j j J ? ? 1 1 (2) where ?j is the standard deviation of yj in the control group and ?1j is the coefficient of interest for outcome j. (Specifically, for the reduced form in equation (1), the ?1j ’s correspond to the ?1j’s in equation (1)).20 In order to account for covariance in the estimates of ?1j / ?j we estimate pooled OLS for all outcomes j ? J. An important limitation of standardized treatment effects is that they implicitly “weight” each outcome within a domain equally, which may not be desirable. In addition to standardized treatment effects, in each domain we also report the underlying estimates on the individual outcomes (i.e. the ?1j’s) due to their ease of interpretation, as well as their individual interest. For the individual outcomes we report both per comparison p-values and “family-wise” p-values adjusted to account for the multiple outcomes examined within the domain.21 The per-comparison pvalue may be appropriate if one is interested in the specific hypothesis of the impact of insurance on a  specific outcome (such as depression). This adjusted p-value is more appropriate for considering the test of the specific outcome as part of a set of tests on all the outcomes in the domain of that standardized treatment effect (such as overall health).22 In practice, it is rare for one of our results to be statistically significant (e.g. p value of less than 0.05) in the per-comparison test and not in the multiple inference adjustment.^ 4.2 Validity of the experimental design Our causal inference rests on the twin assumptions that assignment of the ability to apply for OHP Standard was in fact randomized in the way described, and that the treatment and control individuals in the sub-samples we use to analyze outcomes are not differentially selected from the full sample. The lottery’s random selection process was performed by Oregon’s Department of Human Services (DHS). We verified through independent computer simulations that we could replicate their procedure to within sampling error(Appendix 2.1 and Table A12 provide more detail); we also demonstrate that the procedure we used to draw our survey sample produced balance of treatment and control characteristics (see Appendix 2.2). Differences in attrition (match rates or response rates) or in the pre-randomization characteristics of the treatment and control analysis samples would raise concerns about the key assumptions for causal inference. Table 2 therefore investigates treatment-control balance for three different samples in columns 2 through 4, respectively: the sample universe (which is the sample analyzed in the hospital discharge  data and the mortality data), the credit report subsample, and the survey respondents. A priori we were most concerned about the potential for imbalance between treatment and controls in the subsample of survey respondents, given the 50 percent non-response rate (compared to an effective match rate of over 97 percent in the credit report data).23 Panel A shows the balance of match rates. In the credit report data, the difference in (unconditional) match rates between treatment and control is a statistically insignificant 0.4 percentage points. In the survey respondent sample, there is a statistically significant 1.6 percentage point (standard error = 0.7) lower response rate for treated individuals, off of a 51 percent base.24 By way of comparison, our estimated difference in response rates across treatment and control is much smaller than in the RAND Health Insurance Experiment; there, the overall response rate was higher (77 percent), but those randomized into more coverage had systematically higher response rates, with a 24 percentage point difference in response rate between the most and least comprehensive plan (87 percent versus 63 percent; Newhouse et al., 1993). Among the matched or responding subsample, Panel B reports the treatment-control balance for various pre-randomization characteristics. The first row (“lottery list characteristics”) shows the Fstatistics and p-values on the treatment-control balance of all the nine lottery list demographics. The second row (“pre-randomization outcomes”) shows the balance of pre-randomization outcomes that match ones that we subsequently analyze post-randomization. The number and definition of these outcomes varies according to the sample studied.25 Finally, the third row (“both of the above”) examines balance on the lottery list and pre-randomization outcomes combined. In each of the three samples we are unable to reject the null of treatment-control balance on the lottery list variables, the pre-randomization measures, or the combined set of variables. All p-values are above 0.23. For the survey respondent subsample – where a priori we were most concerned about potential imbalance – the p-values are all at least 0.28.26 A separate question from the balance of treatment and controls within a subsample is how the various subsamples compare to each other. Differences in characteristics across sub-samples do not threaten causal inference, but may be important for comparing estimates across data sets as well as for extrapolatingto other contexts. Appendix Table A1 shows that survey respondents are on average almost two years older and about 3 percentage points more likely to be female than the full sample. There are no obvious observable differences between the credit report subsample and the full sample. 4.3 IV The reduced form (or intent-to-treat) estimates from equation (1) provide an estimate of the causal effect of winning the lottery (i.e. winning the ability to apply for OHP Standard). This provides an estimate of the net impact of expanding access to public health insurance. We are also interested in the impact of insurance coverage. INSURANCE is a measure of insurance coverage and all other variables are as defined in equation (1). We estimate equation (3) by two stage least squares (2SLS), using the following first stage equation: ihj LOTTERYih Xih Vih ihj INSURANCE ? ? 0 ? ? 1 ? ? 2 ? ? 3 ? ? (4) in which the excluded instrument is the variable LOTTERY with the first stage coefficient of ?1. 27 Because the model is just identified, the 2SLS estimate of ?1 is given by the ratio of the reduced form (equation 1) and first stage (equation 4) coefficients (?1/ ?1). We interpret the 2SLS estimates as a local average treatment effect or LATE. (Imbens and Angrist, 1994). In other words, the 2SLS estimate of ?1 identifies the causal impact of insurance among the subset of individuals who would obtain insurance on winning the lottery and would not obtain insurance without winning the lottery (i.e. the compliers). Table 3 reports our first stage estimates based on estimation of equation (4). The first row reports the estimates for the measure of insurance that is used in all of our IV estimates: whether the individual was ever on Medicaid (which includes both OHP Standard and OHP Plus) during our study period, as measured by the state’s Medicaid enrollment files.28 The results indicate a first stage of 0.26 for both the full sample (column 2) and the credit-report subsample (column 4) and a first stage of 0.29 for the survey respondents (column 6). All of these first stages have F-statistics above 500. The first stage coefficient is considerably less than 1, primarily reflecting the 30 percent take-up discussed above; in addition, a small percentage of the controls became eligible for OHP Plus over our study period.29 Relative to our study population, compliers are somewhat older, more likely white, in worse health, and in lower socio-economic status (see Appendix table A23). The subsequent rows of Table 3 report first stage estimates for alternative definitions of insurance. We focus the discussion on the results for the full sample (column 2); the other samples show very similar patterns. Not surprisingly, the results in row 2 show that estimated first stage is virtually the same if we examine only coverage by OHP Standard (the program directly affected by the lottery) versus coverage by either public program, indicating that selection by the lottery is not associated with an increase in coverage by OHP Plus. Row 3 shows that on average the lottery is associated with an increase of 3.4 months with Medicaid. Over time the difference in insurance coverage between treatment and controls attenuates as enrolled individuals had to recertify their eligibility every 6 months and control individuals could find other insurance (particularly OHP Plus); this can be seen clearly in Appendix Figure A3. As a result, when the dependent variable is defined as “on Medicaid at the end of the study period” rather than our primary measure of “ever on Medicaid” the estimated first stage declines from 0.26 (row 1) to 0.15 (row 4). The survey data (column 4) provide a broader measure of insurance coverage than available in the Medicaid administrative data. The results in row 5 indicate that the estimated increase in self-reported “any” insurance coverage is very similar (within two percentage points) to the estimated increase in selfreported Medicaid coverage (row 7). Consistent with this, row 6 indicates a decline in self reports of private insurance coverage of only 0.8 percentage points (standard error =0.5). The estimated increase in Medicaid coverage as reported in the survey and as measured in the administrative data are quite similar when measured over the same time horizon (compare rows 7 and 8). The identifying assumption behind the 2SLS estimates is that there is no effect on the outcomes studied, on average, of winning the lottery that does not operate via the lottery’s impact on insurance coverage. We believe this is a reasonable approximation, but may not be strictly true. There are (at least) two possible types of violations. First, it is possible that the event of winning (or losing) the lottery may have direct effects on the outcomes we study, although it seems unlikely to us that such effects both exist and would persist for a year after the lottery. 30 Second, individuals who apply for public health insurance may also be encouraged to apply for other public programs for which they are eligible, such as food stamps or cash welfare. In particular, if the individual applied for OHP in person (rather than by mail) case workers were instructed to offer assistance to the applicant (if they were interested) in also applying for TANF (cash welfare), the Supplemental Nutrition Assistance Program (more commonly known as food stamps), and employment related child care assistance. These other cash (or cash equivalent) transfer programs could have direct effects on the outcomes we study. This is not an idiosyncratic feature of our setting but a more general feature of the application process for public programs; as such, it may be a relevant component of the impact of attempts to expand Medicaid more generally. However, any direct impact of winning the lottery on receipt of other benefits is a violation of the exclusion restriction for the IV interpretation of the impact of insurance per se, as opposed to the effect of expanded access to Medicaid (the reduced form). The results in Table 3 suggest that there is a statistically significant but substantively trivial impact of the treatment (lottery selection) on other program receipts. Rows 9 and 10 of Table 3 indicate that selection by the lottery is associated with a statistically significant increase in the probability of food stamp receipt of 1.7 percentage points and of total food stamp benefits over about $60 over a 16 month period, or less than 0.5 percent of annual income.31 Estimates of the income elasticity of health care use range from a low end of about 0 to a high end of about 1.5, suggesting that the income effect of food stamp receipt on health care use would be considerably less than 1 percent. The impact on health seems likely to be negligible as well.