Tag: standardized tests

Want Accountability in Education? Empower Parents

The selection of Betsy DeVos for Secretary of Education has exposed longstanding tensions among education reformers who are united in their support for expanding educational choice but divided over the government’s role in regulating such programs.

The schism is often portrayed as being between those who support or reject “accountability,” but this isn’t quite accurate. The real disagreement is not whether there should be accountability, but to whom schools should be held accountable: parents or bureaucrats. As Lindsey Burke and I argue in a new report published by the Heritage Foundation and the Texas Public Policy Foundation, educational choice programs like education savings accounts should place the accountability for academic outcomes with parents.

For decades, the term “accountability” primarily referred, in education policy circles, to government regulations intended to ensure quality. That’s because most children attend their assigned district schools, which are not directly answerable to parents and function as de facto monopolies. As Lindsey and I explain:

A distinctive feature of monopolies is lack of accountability. Because district schools are not held directly accountable to parents, some policymakers have attempted to impose accountability through top-down government regulations. Yet decades of attempts to regulate district schools into quality have had little effect. Unfortunately, too many policymakers have still come to see centralized government regulations as synonymous with “accountability” rather than an inferior alternative to direct accountability to parents, and have therefore sought to impose similar regulations on choice programs. However, regulations designed for a monopoly system are inappropriate for a market-based system.

In a market-based system, producers are held directly accountable to consumers for results. The government sets certain rules against fraud or health and safety standards, but the consumers ultimately decide whether a product or service meets their needs. Likewise, the government could ensure that ESA funds are spent on qualifying educational products and services, but the accountability for results should lie with parents, who are in the best position to evaluate those results. Holding education providers directly accountable to parents creates a feedback loop that does not exist in more centralized, top-down systems like the district schools. As social scientist Yuval Levin has argued, this enables the system to “channel social knowledge from the bottom up rather than…impose technical knowledge from the top down.” This channeling is accomplished “through a process of experimentation, evaluation, and evolution.”

If we want an education system that makes significant improvements in quality over time, education providers must be free to innovate and parents must be free to choose the providers that work best for their own children. This system evolves over time because higher-quality providers will attract more parents and lower-quality providers will face pressure to either improve or shut down.

However, technocratic attempts to guarantee quality through imposing uniform standards can interrupt this evolutionary process.

The Price of Technocratic Accountability

The technocratic approach to accountability requires that all schools are judged according to uniform metrics, therefore the technocrats rely heavily (indeed, almost exclusively) on standardized test scores, particularly in math and language arts. The technocratic reformers want to use these scores to set a minimum standard, meaning “underperforming” schools would be excluded from receiving voucher funds–or, in the case of charter schools, be shut down entirely–even against the will of parents who still want to enroll their children there.

Let us be clear about what is at stake. The technocratic approach would eliminate a family’s least-bad educational alternative, leaving children worse off “for their own good.” For example, parents may have chosen a private or charter school that did not perform well on the state’s standardized test overall, but the school may have provided a safer environment than the local district school. Or perhaps the school was succeeding at its mission to aid the most at-risk students, but the state’s uniform “accountability” system failed to take its mission into account. The damage done to children who lose the opportunity to attend schools that their parents believe are better than the alternative is incalculable.

We should also be realistic about the unintended consequences of over-reliance on test scores. Although standardized tests can provide parents with useful information about their child’s academic performance, using them to impose uniform standards that so narrowly define “quality” creates perverse incentives that narrow the curriculum, stifle innovation, and can drive away quality schools from participating in the choice program. As Lindsey and I explain:

When schools are rewarded or punished based on their students’ performance on math and reading tests, they have a strong incentive to divert their time and resources to tested subjects and away from others. A study by the Center on Education Policy found that the time district schools spent on subjects besides math and reading declined considerably after Congress enacted the No Child Left Behind Act (NLCB), which mandated that states require district schools to administer the state standardized math and reading tests in grades three through eight and report the results. In the five years after NCLB was implemented, approximately 62 percent of elementary district schools reported increasing the amount of time spent on English language arts and/or math, and 44 percent reported decreasing time spent on social studies, science, art and music, physical education, lunch, or recess.

The narrowing curriculum is particularly alarming because, as Jay P. Greene has noted, recent research has found that “later success in math, reading, and science depends on early acquisition of the kind of ‘general knowledge’ and fine-motor skills learned through art and other subjects.” In other words, a narrower curriculum not only deprives students of having a broader and more enriching education, but also negatively impacts their performance in the tested subjects. “If we narrow education to the mechanics of math and reading as captured by yearly testing,” Greene concludes, “we short-change the broader knowledge that is the key to academic success later.”

Mandating a single test exacerbates this phenomenon. Within the tested subjects, schools have a strong incentive to teach the concepts that will be on the mandated test. This incentive to “teach to the test” can result in a de facto curriculum. For example, if a school had been teaching math concepts A, B, and C in grade 7, but the new state test was going to cover concepts B, D, and E, the school would almost certainly drop concepts A and C in favor of D and E, even if the math teachers believe that the original curriculum was superior. Keeping the original curriculum would put their students at a disadvantage on the state test vis-à-vis students at other schools that had aligned their curriculum to the test. This standardization might make sense in a world in which there was one right way to teach math, or at least one right order to teach concepts, but that is not the case.

Again, this isn’t to say that we should do away with testing entirely. As Robert Pondiscio recently wrote, standardized tests should be “used to illuminate and inform parent choice, but not to limit it.” Tests can provide valuable information, but using the tests as the sole or primary metric of performance does more harm than good. What’s needed is a more comprehensive understanding of quality that considers the needs of individual students, not just the “typical” student, and that’s something that parents are in a much better position to determine than technocrats.

The Common Coring of Private Schools

Today, the Fordham Institute released a “policy toolkit” proposing private schools be required to administer state tests to all students participating in a school choice program, and publicize the results. Private schools that the state deemed persistently underperforming would be expelled from the program. Fordham argues that such measures have the potential to raise student achievement and provide parents with the information needed to make better decisions about their children’s education. Though Fordham’s plan is well-intentioned, their justifications are unpersuasive and their proposal is more likely to do harm than good.

Little Evidence to Support a Testing Mandate for Private Schools

First, there is scant evidence to support Fordham’s claim that test-based accountability measures “may boost student achievement.” Fordham rests its claim on the results of but a single year in a single study of a single school choice program: the final year of the School Choice Demonstration Project’s five-year analysis of the Milwaukee voucher program.

During the first four years of the study, voucher students took a low-stakes test, but in the final year of the study, policymakers increased the stakes by mandating that the test results be publicized and the scores improved. Fordham argues that this proves that high-stakes testing improves performance but one of the study’s authors, Dr. Patrick Wolf of the University of Arkansas, has previously cautioned the Fordham Institute against reading too much into that finding, calling it “enticing and suggestive but hardly conclusive”:

As we point out in the report, it is entirely possible that the surge in the test scores of the voucher students was a “one-off” due to a greater focus of the voucher schools on test preparation and test-taking strategies that year.  In other words, by taking the standardized testing seriously in that final year, the schools simply may have produced a truer measure of student’s actual (better) performance all along, not necessarily a signal that they actually learned a lot more in the one year under the new accountability regime.

But even if the was no question that the higher test scores actually reflected increased performance, it would still only be one study. When Fordham cited this study as support for its proposal six months ago, Andrew J. Coulson responded:

A single study, no matter how carefully executed, is not a scientific basis for policy. Because a single study is not science. Science is a process of making and testing falsifiable predictions. It is about patterns of evidence. Bodies of evidence. Fordham offers only a toe.

By contrast, there is a significant body of evidence that school choice programs work without Fordham’s sought-after government regulation. Of twelve randomized controlled trials—the gold standard of social science research—eleven found that school choice programs improve outcomes for some or all students while only one found no statistically significant difference and none found a negative impact. None of these school choice programs studied were designed along the lines of the Fordham proposal.

In fact, Fordham’s preferred policy is undermined by a large body of evidence. A 2009 literature review of the within-country studies comparing outcomes among different types of school systems worldwide revealed that the most market-like and least regulated education systems tended to produce student outcomes superior to more heavily regulated systems, including those with a substantial number state-funded and regulated private schools. In short, the best form of accountability is directly to parents, not government bureaucrats and their tests.

Why School Choice Programs Should Not Require Testing

As Georgia’s legislature considers a bill to expand the state’s scholarship tax credit (STC) program and increase transparency, some are calling for additional regulations. Writing in yesterday’s Atlanta Journal-Constitution, Adam Emerson of the Thomas B. Fordham Institute argued that the Peach State should require private schools that accept scholarship students to administer standardized tests. That would be a mistake.

Emerson argues that parents, educators, and policymakers “deserve to compare the gains that students make in different school environments.” Test scores can be a useful, albeit limited and incomplete, method of comparing the academic effectiveness of different schools. Under the current system, parents are free to choose private schools that administer standardized tests and avoid those that do not or vice-versa. Likewise, donors are free to direct their money to scholarship organizations that only fund schools that do or do not administer standardized tests.

By contrast, a testing mandate would severely limit or even eliminate these choices. Parents who are concerned about the “teach to the test” phenomenon or whose child reacts negatively to testing, and educators who believe that standardized testing gets in the way of real learning will have little to no choice but to participate in the standardized testing regime, as even Emerson agrees.

Emerson argues that the vast majority of private schools would still participate in the STC program even with a standardized testing mandate. He cites a recent Fordham study of the impact of regulation in school choice programs showing that “only 3 percent of non-participating schools cited governmental regulations as the most important reason to opt out.” That’s exactly the problem. The mandate would force private schools to choose between eschewing both the tests and all the students who need school choice scholarships to attend their school, or accepting both. Unsurprisingly, the vast majority of schools succumb to the financial pressure.

Researchers tend to support testing mandates because they allow them to evaluate the effectiveness of certain reforms, at least to an extent. But the very act of measurement itself can distort the very thing they are trying to measure. Even aside from the cheating scandals, the tests create strong incentives for teachers to teach differently in order that their students perform well on the test.

Of course, some will argue, “that’s not a bug, that’s a feature!” They support standardized tests as a means of increasing accountability. “If you want to move something, you have to measure it,” they argue, and they have a point. But standardized tests are not the only way to measure learning. Supporters of standardized testing know that, but they want a uniform measures so that they can compare apples to apples. The problem is that uniform measures create a powerful incentive to move toward uniform behavior. As James Shuls of the Show-Me Institute wrote recently:

The fact is that curriculum standards don’t tell teachers how to teach in the same way that a high jump bar doesn’t tell a jumper how to jump. You could theoretically jump over a high jump bar in whatever way you would like; but because of how the jump is structured there is a clear advantage to doing the old Fosbury Flop.

While standardized tests are not as imposing as curriculum standards, what’s on the test can drive what is taught in the classroom, when it is taught, and how it is taught. Professor Jay P. Greene argued along similar lines regarding national standards:

Such uniformity would only make sense if: 1) there was a single best way for all students to learn; 2) we knew what it was; 3) we could be sure the people running this nationalized education system would adopt that correct approach; and 4) they would remain in charge far into the future. But that isn’t how things are. There is no consensus on what all students need to know. Different students can best be taught and assessed in different ways.

Standardized tests create an incentive for uniformity when we should be fostering a diversity of traditional and innovative pedagogical methods. To the extent that researchers, policymakers and some educators believe that standardized tests are useful, they should make their case in the free marketplace of ideas and encourage parents to choose the schools that administer them. What they shouldn’t do is use the coercive power of government to fashion a top-down system of testing that could squelch diversity and innovation. 

South Dakota: Second State to Ignore NCLB Requirements

South Dakota joined Idaho this week in declaring that it will not raise its student proficiency targets next year as required by the NCLB. Under the law, states have been required to bring increasing percentages of their students up to the “proficient” level on their own tests. By 2014, NCLB demands that all students be deemed proficient by their respective state departments of education.

The belief driving NCLB was that, if we we raise government standards for what students are supposed to know and be able to do, they will learn more. They haven’t, according to the best, nationally representative indicator of academic outcomes: the NAEP Long Term Trends tests. By the end of high school, overall student achievement is no better today than it was 40 years ago. In science, it’s slightly worse.

The reason NCLB failed is that its core belief was and is wrong: external, government-mandated standards are not the driving force of progress. It is the freedom and incentives of competitive marketplaces that drive up performance and productivity. I’ve already made this case in the context of the national education standards movement, and the same arguments and evidence apply to NCLB.

The testing component of NCLB was never more than a thermometer—and a broken, unreliable thermometer at that; allowing states to play games with test difficulty and the definition of “proficiency” in order to massage their results.

Thermometers don’t cure people. They are at best a diagnostic tool.

If we want to see the same kind of progress, productivity growth, and innovation in education that we’ve come to expect in every other field, we have one choice and one choice only: adopt the same freedoms and incentives in education that have driven progress in other fields. Either we allow education to benefit from the free enterprise system or we should get used to disappointment.

We Must Protect This Failing House! (And To Heck With the Kids In It)

The New York Times’ “Room for Debate” website is once again hosting a forum on education, to which I have contributed some thoughts. The topic: whether there should be federal tax credits for home schoolers.

I won’t rehash my contribution – obviously, you can read it right on the site – but I wanted to respond quickly to two other entries.

The first is from Chester Finn, president of our favorite conservative sparring partner in education, the Thomas B. Fordham Instititute. I just want to thank him for substantiating a warning I offer in my contribution: Create federal home-schooling credits and don’t be surprised if you also get requirements that home schoolers be judged on stultifying standardized tests.  It’s exactly what Finn calls for:

In return for the financial help, however, home-schooled students should be required to take state tests, just as they would do in regular school, charter school or virtual schools. And if they don’t pass those tests, either the subsidy vanishes or the kids must enroll in some sort of school with a decent academic track record.

The second person I want to respond to is former Bush II official Susan Neuman, who generally offers the right advice by warning even more starkly than I did that home schoolers demanding tax credits are making a deal with the regulatory devil. That’s fine, as is her reporting that by what indications we have “children who have been home-schooled do remarkably well, attending well-respected colleges and universities and going on to successful careers.” Unfortunately, all that was preceded by the Reductio ad Hitlerum of education debates: Smearing any effort to even the playing field between public schools and other educational arrangements as an “attempt … to destroy public education.”

I know that this will never catch on with people determined to preserve public schools’ near-monopoly on tax dollars no matter how well other arrangements actually educate children (not to mention serve taxpayers and society overall), but it is time to stop treating public education as if it is synonymous with public schools! Indeed, you demonstrate more dedication to public education if you fight to get kids access to the best education wherever it is offered than if you make your ultimate goal preserving government schools. Yet the monopoly defenders insist on smearing choice advocates as being at war with public education.

Stop with this trashy tactic. Wanna say supporters of educational choice are at war with public schools? Fine. But with public education? Sorry – if anything, they’re the ones truly fighting to get the best possible education for all.

Winters’ Content Standards — Can they Work?

Marcus Winters offers a clever new national standards proposal in the current Education Week: reward states whose students do well on their own standards _and_ whose standards prove challenging to students from other states. Winters suggests administering each state’s standardized tests to random, nationally representative samples of students to determine how challenging they are. The federal government would then give the greatest amount of funding to states whose students perform well on tests that prove challenging to kids around the country.

This system would be gamed. The way to “win” would be to develop highly detailed, easy, obscure standards. Literature would consist of detailed analysis of the early works of Nathanial Hawthorne, math would focus on theorems not normally covered but not overly challenging, history would focus on seldom-told tales of the host state or the nation or world. The host state would then teach intensely to these specialized standards, knowing that its own students could master them and students in other states – receiving a completely different curriculum – would perform poorly. It would be neither a “race to the top” nor a “race to the bottom,” but rather a “race to the trivial.”

This proposal also suffers the same problem that a single set of national standards would suffer: it would force all students of a given age to march through their state’s curriculum at the same pace, denying the obvious reality that kids of the same age learn the various subjects at different paces. Shackling them together into a scholastic chain-gang is not sound pedagogy.

What is encouraging about this proposal, though, is that it attempts to marshal both competition and incentives in pursuit of improved performance. Clearly, it’s on the right track. But why reinvent the wheel? We already HAVE a system that has proven, over centuries, to be able to effectively combine competition, freedom, and incentives in pursuit of innovation and excellence: the free enterprise system.

School systems organized along free market lines dramatically outperform all others – especially those which are most closely overseen, and run, by the state. We just need to figure out how to bring a free and competitive education marketplace within reach of all students.