This week, researchers Patrick J. Wolf, M. Danish Shakeel, and Kaitlin P. Anderson of the University of Arkansas released the results of their painstaking meta-analysis of the international, gold-standard research on school choice programs, which concluded that, on average, such programs have a statistically significant positive impact on student performance on reading and math tests. Moreover, the magnitude of the positive impact increased the longer students participated in the program.
As Wolf observed in a blog post explaining the findings, the “clarity of the results… contrasts with the fog of dispute that often surrounds discussions of the effectiveness of private school choice.”
That’s So Meta
One of the main advantages of a meta-analysis is that it can overcome the limitations of individual studies (e.g., small samples sizes) by pooling the results of numerous studies. This meta-analysis is especially important because it includes all random-assignment studies on school choice programs (the gold standard for social science research), while excluding studies that employed less rigorous methods. The analysis included 19 studies on 11 school choice programs (including government-funded voucher programs as well as privately funded scholarship programs) in Colombia, India, and the United States. Each study compared the performance of students who had applied for and randomly won a voucher to a “control group” of students who had applied for a voucher but randomly did not receive one. As Wolf explained, previous meta-analyses and research reviews omitted some gold-standard studies and/or included less rigorous research:
The most commonly cited school choice review, by economists Cecilia Rouse and Lisa Barrow, declares that it will focus on the evidence from existing experimental studies but then leaves out four such studies (three of which reported positive choice effects) and includes one study that was non-experimental (and found no significant effect of choice). A more recent summary, by Epple, Romano, and Urquiola, selectively included only 48% of the empirical private school choice studies available in the research literature. Greg Forster’s Win-Win report from 2013 is a welcome exception and gets the award for the school choice review closest to covering all of the studies that fit his inclusion criteria – 93.3%.
Survey Says: School Choice Improves Student Performance
The meta-analysis found that, on average, participating in a school choice program improves student test scores by about 0.27 standard deviations in reading and 0.15 standard deviations in math. In laymen’s terms, these are “highly statistically significant, educationally meaningful achievement gains of several months of additional learning from school choice.”