Tag: standardized tests

The Common Coring of Private Schools

Today, the Fordham Institute released a “policy toolkit” proposing private schools be required to administer state tests to all students participating in a school choice program, and publicize the results. Private schools that the state deemed persistently underperforming would be expelled from the program. Fordham argues that such measures have the potential to raise student achievement and provide parents with the information needed to make better decisions about their children’s education. Though Fordham’s plan is well-intentioned, their justifications are unpersuasive and their proposal is more likely to do harm than good.

Little Evidence to Support a Testing Mandate for Private Schools

First, there is scant evidence to support Fordham’s claim that test-based accountability measures “may boost student achievement.” Fordham rests its claim on the results of but a single year in a single study of a single school choice program: the final year of the School Choice Demonstration Project’s five-year analysis of the Milwaukee voucher program.

During the first four years of the study, voucher students took a low-stakes test, but in the final year of the study, policymakers increased the stakes by mandating that the test results be publicized and the scores improved. Fordham argues that this proves that high-stakes testing improves performance but one of the study’s authors, Dr. Patrick Wolf of the University of Arkansas, has previously cautioned the Fordham Institute against reading too much into that finding, calling it “enticing and suggestive but hardly conclusive”:

As we point out in the report, it is entirely possible that the surge in the test scores of the voucher students was a “one-off” due to a greater focus of the voucher schools on test preparation and test-taking strategies that year.  In other words, by taking the standardized testing seriously in that final year, the schools simply may have produced a truer measure of student’s actual (better) performance all along, not necessarily a signal that they actually learned a lot more in the one year under the new accountability regime.

But even if the was no question that the higher test scores actually reflected increased performance, it would still only be one study. When Fordham cited this study as support for its proposal six months ago, Andrew J. Coulson responded:

A single study, no matter how carefully executed, is not a scientific basis for policy. Because a single study is not science. Science is a process of making and testing falsifiable predictions. It is about patterns of evidence. Bodies of evidence. Fordham offers only a toe.

By contrast, there is a significant body of evidence that school choice programs work without Fordham’s sought-after government regulation. Of twelve randomized controlled trials—the gold standard of social science research—eleven found that school choice programs improve outcomes for some or all students while only one found no statistically significant difference and none found a negative impact. None of these school choice programs studied were designed along the lines of the Fordham proposal.

In fact, Fordham’s preferred policy is undermined by a large body of evidence. A 2009 literature review of the within-country studies comparing outcomes among different types of school systems worldwide revealed that the most market-like and least regulated education systems tended to produce student outcomes superior to more heavily regulated systems, including those with a substantial number state-funded and regulated private schools. In short, the best form of accountability is directly to parents, not government bureaucrats and their tests.

Why School Choice Programs Should Not Require Testing

As Georgia’s legislature considers a bill to expand the state’s scholarship tax credit (STC) program and increase transparency, some are calling for additional regulations. Writing in yesterday’s Atlanta Journal-Constitution, Adam Emerson of the Thomas B. Fordham Institute argued that the Peach State should require private schools that accept scholarship students to administer standardized tests. That would be a mistake.

Emerson argues that parents, educators, and policymakers “deserve to compare the gains that students make in different school environments.” Test scores can be a useful, albeit limited and incomplete, method of comparing the academic effectiveness of different schools. Under the current system, parents are free to choose private schools that administer standardized tests and avoid those that do not or vice-versa. Likewise, donors are free to direct their money to scholarship organizations that only fund schools that do or do not administer standardized tests.

By contrast, a testing mandate would severely limit or even eliminate these choices. Parents who are concerned about the “teach to the test” phenomenon or whose child reacts negatively to testing, and educators who believe that standardized testing gets in the way of real learning will have little to no choice but to participate in the standardized testing regime, as even Emerson agrees.

Emerson argues that the vast majority of private schools would still participate in the STC program even with a standardized testing mandate. He cites a recent Fordham study of the impact of regulation in school choice programs showing that “only 3 percent of non-participating schools cited governmental regulations as the most important reason to opt out.” That’s exactly the problem. The mandate would force private schools to choose between eschewing both the tests and all the students who need school choice scholarships to attend their school, or accepting both. Unsurprisingly, the vast majority of schools succumb to the financial pressure.

Researchers tend to support testing mandates because they allow them to evaluate the effectiveness of certain reforms, at least to an extent. But the very act of measurement itself can distort the very thing they are trying to measure. Even aside from the cheating scandals, the tests create strong incentives for teachers to teach differently in order that their students perform well on the test.

Of course, some will argue, “that’s not a bug, that’s a feature!” They support standardized tests as a means of increasing accountability. “If you want to move something, you have to measure it,” they argue, and they have a point. But standardized tests are not the only way to measure learning. Supporters of standardized testing know that, but they want a uniform measures so that they can compare apples to apples. The problem is that uniform measures create a powerful incentive to move toward uniform behavior. As James Shuls of the Show-Me Institute wrote recently:

The fact is that curriculum standards don’t tell teachers how to teach in the same way that a high jump bar doesn’t tell a jumper how to jump. You could theoretically jump over a high jump bar in whatever way you would like; but because of how the jump is structured there is a clear advantage to doing the old Fosbury Flop.

While standardized tests are not as imposing as curriculum standards, what’s on the test can drive what is taught in the classroom, when it is taught, and how it is taught. Professor Jay P. Greene argued along similar lines regarding national standards:

Such uniformity would only make sense if: 1) there was a single best way for all students to learn; 2) we knew what it was; 3) we could be sure the people running this nationalized education system would adopt that correct approach; and 4) they would remain in charge far into the future. But that isn’t how things are. There is no consensus on what all students need to know. Different students can best be taught and assessed in different ways.

Standardized tests create an incentive for uniformity when we should be fostering a diversity of traditional and innovative pedagogical methods. To the extent that researchers, policymakers and some educators believe that standardized tests are useful, they should make their case in the free marketplace of ideas and encourage parents to choose the schools that administer them. What they shouldn’t do is use the coercive power of government to fashion a top-down system of testing that could squelch diversity and innovation. 

South Dakota: Second State to Ignore NCLB Requirements

South Dakota joined Idaho this week in declaring that it will not raise its student proficiency targets next year as required by the NCLB. Under the law, states have been required to bring increasing percentages of their students up to the “proficient” level on their own tests. By 2014, NCLB demands that all students be deemed proficient by their respective state departments of education.

The belief driving NCLB was that, if we we raise government standards for what students are supposed to know and be able to do, they will learn more. They haven’t, according to the best, nationally representative indicator of academic outcomes: the NAEP Long Term Trends tests. By the end of high school, overall student achievement is no better today than it was 40 years ago. In science, it’s slightly worse.

The reason NCLB failed is that its core belief was and is wrong: external, government-mandated standards are not the driving force of progress. It is the freedom and incentives of competitive marketplaces that drive up performance and productivity. I’ve already made this case in the context of the national education standards movement, and the same arguments and evidence apply to NCLB.

The testing component of NCLB was never more than a thermometer—and a broken, unreliable thermometer at that; allowing states to play games with test difficulty and the definition of “proficiency” in order to massage their results.

Thermometers don’t cure people. They are at best a diagnostic tool.

If we want to see the same kind of progress, productivity growth, and innovation in education that we’ve come to expect in every other field, we have one choice and one choice only: adopt the same freedoms and incentives in education that have driven progress in other fields. Either we allow education to benefit from the free enterprise system or we should get used to disappointment.

We Must Protect This Failing House! (And To Heck With the Kids In It)

The New York Times’ “Room for Debate” website is once again hosting a forum on education, to which I have contributed some thoughts. The topic: whether there should be federal tax credits for home schoolers.

I won’t rehash my contribution – obviously, you can read it right on the site – but I wanted to respond quickly to two other entries.

The first is from Chester Finn, president of our favorite conservative sparring partner in education, the Thomas B. Fordham Instititute. I just want to thank him for substantiating a warning I offer in my contribution: Create federal home-schooling credits and don’t be surprised if you also get requirements that home schoolers be judged on stultifying standardized tests.  It’s exactly what Finn calls for:

In return for the financial help, however, home-schooled students should be required to take state tests, just as they would do in regular school, charter school or virtual schools. And if they don’t pass those tests, either the subsidy vanishes or the kids must enroll in some sort of school with a decent academic track record.

The second person I want to respond to is former Bush II official Susan Neuman, who generally offers the right advice by warning even more starkly than I did that home schoolers demanding tax credits are making a deal with the regulatory devil. That’s fine, as is her reporting that by what indications we have “children who have been home-schooled do remarkably well, attending well-respected colleges and universities and going on to successful careers.” Unfortunately, all that was preceded by the Reductio ad Hitlerum of education debates: Smearing any effort to even the playing field between public schools and other educational arrangements as an “attempt … to destroy public education.”

I know that this will never catch on with people determined to preserve public schools’ near-monopoly on tax dollars no matter how well other arrangements actually educate children (not to mention serve taxpayers and society overall), but it is time to stop treating public education as if it is synonymous with public schools! Indeed, you demonstrate more dedication to public education if you fight to get kids access to the best education wherever it is offered than if you make your ultimate goal preserving government schools. Yet the monopoly defenders insist on smearing choice advocates as being at war with public education.

Stop with this trashy tactic. Wanna say supporters of educational choice are at war with public schools? Fine. But with public education? Sorry – if anything, they’re the ones truly fighting to get the best possible education for all.

Winters’ Content Standards — Can they Work?

Marcus Winters offers a clever new national standards proposal in the current Education Week: reward states whose students do well on their own standards _and_ whose standards prove challenging to students from other states. Winters suggests administering each state’s standardized tests to random, nationally representative samples of students to determine how challenging they are. The federal government would then give the greatest amount of funding to states whose students perform well on tests that prove challenging to kids around the country.

This system would be gamed. The way to “win” would be to develop highly detailed, easy, obscure standards. Literature would consist of detailed analysis of the early works of Nathanial Hawthorne, math would focus on theorems not normally covered but not overly challenging, history would focus on seldom-told tales of the host state or the nation or world. The host state would then teach intensely to these specialized standards, knowing that its own students could master them and students in other states – receiving a completely different curriculum – would perform poorly. It would be neither a “race to the top” nor a “race to the bottom,” but rather a “race to the trivial.”

This proposal also suffers the same problem that a single set of national standards would suffer: it would force all students of a given age to march through their state’s curriculum at the same pace, denying the obvious reality that kids of the same age learn the various subjects at different paces. Shackling them together into a scholastic chain-gang is not sound pedagogy.

What is encouraging about this proposal, though, is that it attempts to marshal both competition and incentives in pursuit of improved performance. Clearly, it’s on the right track. But why reinvent the wheel? We already HAVE a system that has proven, over centuries, to be able to effectively combine competition, freedom, and incentives in pursuit of innovation and excellence: the free enterprise system.

School systems organized along free market lines dramatically outperform all others – especially those which are most closely overseen, and run, by the state. We just need to figure out how to bring a free and competitive education marketplace within reach of all students.