The most recent federal evaluation of the Washington, D.C., school voucher program found that it led to a 10‐point reduction in math test scores and a statistically insignificant change in reading test scores after two years. Because of this evidence, some people are now calling for more standardized testing regulations of the private schools that low‐income families want their kids to attend.
This is a very bad idea.
The program already requires that participating students take the D.C. Public Schools assessment in grades 3 through 8. More stringent test score regulations would increase the schools’ costs of participation — meaning new rules and expenses associated with complying with them — and therefore discourage them from accepting vouchers, reducing the number of private school options available to families. Limiting the number of options guarantees that many children from low‐income families would end in up schools that were not their top choices.
Families choose schools for their children based on several important factors, including culture, individual attention, and, of course, safety. Research tells us that parents — unsurprisingly — often value these things more than standardized test scores. Perhaps that is why the same federal evaluation of the D.C. program found that 74 percent of voucher parents reported that their child’s school was “very safe,” while only about 55 percent of those not offered a voucher reported their child was at a “very safe” school.
Restricting the abilities of families to choose safer schools because of math test scores could force disadvantaged children to attend schools that their families perceive as being more dangerous. Though an unintended consequence, that would be a huge mistake.
But the lackluster math test score result brings up the question: Why don’t families care all that much about standardized exams?
It may just be that test scores aren’t as important as the experts wish them to be. A recent review of the school choice evidence released by the American Enterprise Institute found that test scores are weak predictors of long‐term outcomes such as high school graduation and college enrollment. In fact, the authors of the previous peer‐reviewed evaluation of the D.C. program found no “conclusive evidence that the [voucher program] affected student achievement,” but, “the program significantly improved students’ chances of graduating from high school” by 21 percentage points.
Similarly, I have identified 11 studies of private schools indicating divergences between test scores and more important long‐term outcomes such as tolerance of others, adult criminal activity, and happiness. Maybe families are on to something that the experts have been missing all along.
It isn’t likely that additional test score regulations of the D.C. voucher program would harm the rich; all students must meet poverty‐level guidelines to be eligible for the program. Families with the financial means would still be able to send their kids to the schools they want. The only difference would be that government regulators would prevent the poor from exercising the same choices for their kids. Why does this suddenly become such a big issue when low‐income families are given the opportunity to choose private schools? And what makes regulators think they know so much more about what other people’s children need?
If test scores were so valuable, we wouldn’t need to restrict school choices based on them. But let’s assume standardized test scores were valid measures. Even if that heroic assumption were true, regulators often use test score levels to try to determine the quality of schools. The obvious problem with this approach is that a school with highly disadvantaged students would appear low‐quality even if it were doing very well at accelerating students’ academic growth. In theory, regulators could use more rigorous methodology to determine the effects of individual schools on student test scores.
But even if central planners were keen enough to use the best econometric tool that we have today — value‐added modeling — they still would not be able to determine which schools were the “bad” ones. The short explanation for this problem is the fact that even this approach assumes that children are randomly assigned to schools, while the concept of school choice guarantees that this crucial assumption is not met. But even if regulators somehow figured out how to get past this methodological problem in the future, because the validity of experiments relies on large samples, the models could never tell us the effects of schools on individual students.
This means that — even if we all agreed that test scores are valuable — even the best tools available to regulators could lead them to unintentionally make good private schools inaccessible to families. Unfortunately, despite their confidence, regulators don’t have magical powers that allow them to mandate good schools and outlaw bad ones. And this is why we must trust all families to make the best educational decisions for their own kids.