Tag: science

Education Policy, The Use of Evidence, and the Fordham Institute

In recent weeks, the Fordham Institute has repeatedly called for government testing and reporting mandates to be imposed on private schools participating in school choice programs (here and here), on the grounds that such “public accountability” improves private school academic outcomes. In defense of this claim, the Fordham Institute cites a study of Milwaukee’s voucher program in which test scores rose following the introduction of such mandates.

Patrick Wolf, director of the research team that conducted the study, has now responded, explaining that his team’s results do not necessarily support Fordham’s claim:

[B]y taking the standardized testing seriously in that final year, the schools simply may have produced a truer measure of student’s actual (better) performance all along, not necessarily a signal that they actually learned a lot more in the one year under the new accountability regime….

What about the encouraging trend that lower-performing schools in the MPCP are being closed down?  [Fordham] mentions that as well and attributes it to the stricter accountability regulations on the program.  That phenomenon of Schumpeterian “creative destruction” pre-dated the accountability changes in the choice program, however, and appears to have been caused mainly by low enrollments in low-performing choice schools, as parents “voted with their feet” against such institutional failure. Sure, the new high-stakes testing and public reporting requirements might accelerate the creative destruction of low-performing choice schools in Milwaukee, but that remains to be seen. [emphasis added]

But there is a deeper problem with the Fordham claim, to which Wolf alludes: a single study, no matter how carefully executed, is not a scientific basis for policy. Because a single study is not science. Science is a process of making and testing falsifiable predictions. It is about patterns of evidence. Bodies of evidence. Fordham offers only a toe.

And Fordham’s preferred policy not only lacks a body of supporting evidence, it is undermined by a large body of evidence. When I reviewed the within-country studies comparing outcomes among different types of school systems worldwide in 2009, I sorted the results into two categories: 1) all studies that compared “public” schools to “private” schools, where those terms were loosely defined; and 2) studies that compared “market” schools to “monopoly” schools. “Market” schools were those paid for at least in part directly by parents and only minimally regulated. “Monopoly” schools were public school systems such as those common in the U.S.

The purpose of these separate categorizations was to see if limited regulation and direct parent funding make a real difference, or if private schools that are paid for entirely by the state and subjected to Fordham’s “public accountability” have the same advantages as their more market-like counterparts.

The result of this breakdown of the literature was stark. Studies looking at truly market-like education systems are twice as consistent in finding a private sector advantage as those looking at “private” schools more broadly construed (and thus including state-funded and regulated private schools).

The pattern of evidence thus seems to contradict Fordham’s belief in the merits of “public accountability” in market education systems. What it favors are policies that promote the rise of minimally regulated education markets in which parents pay at least some of the cost of their own children’s education directly themselves, whenever possible.  That’s just the sort of system likely to arise under education tax credit programs.

Science: ‘All Kids Different’

It didn’t get a lot of attention, but in last week’s State of the Union address President Obama celebrated the spread of national curriculum standards that’s been fueled largely by the federal Race to the Top. Of course, he didn’t actually call them “national standards” because no one is supposed to think that these are de facto federal standards that states have been bribed into adopting. The point, though, was clear to those in the know:

Race to the Top is the most meaningful reform of our public schools in a generation. For less than one percent of what we spend on education each year, it has led over 40 states to raise their standards for teaching and learning. These standards were developed, not by Washington, but by Republican and Democratic governors throughout the country.

Despite the celebration of national standards by both the President and lots of other supporters, there is essentially zero evidence that such standards will produce better educational outcomes.  Much of that has to do with the reality of democratically controlled, government education: Those who would be held accountable for getting kids to high standards have the most clout in education politics, and they naturally fight tough standards. It also has a lot to do with human reality: All kids are different. It’s an inescapable observation for anyone who has ever encountered more than one child, but the national-standards crowd prefers to ignore it.

Maybe science will help them see the light. According to the BBC, new research comparing identical and fraternal twins reveals that genetics – something that exists before standards and schooling – has a lot to do with how much and how quickly someone learns:

The researchers examined the test results of 12-year-old twins - identical and fraternal - in English, maths and science.

They found the identical twins, who share their genetic make-up, did more similarly in the tests than the fraternal twins, who share half their genetic make-up.

The report said: “The results were striking, indicating that even when previous achievement and a child’s general cognitive ability are both removed, the residual achievement measure is still significantly influenced by genetic factors.”

In light of this confirmation of the obvious, isn’t it clear that a single timeline for what all children should know and when they should know it makes little sense? And doesn’t it point to the best system being one that gives kids individualized attention?

Of course it does, but that would require “experts” of all stripes to stop trying to impose their solutions on all children. It would also, ultimately, necessitate a system in which parents would choose what’s best for their children, and educators would specialize in all sorts of different curricula, delivery mechanisms, and teaching techniques.  

Unfortunately, few in the education policy world are willing to adopt that utterly logical – but power relinquishing – solution.

The Shocking Truth: The Scientific American Poll on Climate Change

November’s Scientific American features a profile of Georgia Tech atmospheric scientist Judith Curry,  who has committed the mortal sin of  reaching out to other scientists who hypothesize that global warming isn’t the disaster it’s been cracked up to be.  I have personal experience with this, as she invited me to give a research seminar in Tech’s prestigious School of Earth and Atmospheric Sciences in 2008.  My lecture summarizing the reasons for doubting the apocalyptic synthesis of climate change was well-received by an overflow crowd.

Written by Michael Lemonick, who hails from the shrill blog Climate Central, the article isn’t devoid of the usual swipes, calling her a “heretic„ which is hardly at all true.  She’s simply another hardworking scientist who lets the data take her wherever it must, even if that leads her to question some of our more alarmist colleagues. 

But, as a make-up call for calling attention to Curry, Scientific American has run a poll of its readers on climate change.  Remember that SciAm has been shilling for the climate apocalypse for years, publishing a particularly vicious series of attacks on Denmark’s Bjorn Lomborg’s Skeptical Environmentalist.  The magazine also featured NASA’s James Hansen and his outlandish claims on sea-level rise. Hansen has stated, under oath in a deposition, that a twenty foot rise is quite possible within the next 89 years; oddly, he has failed to note that in 1988 he predicted that the West Side Highway in Manhattan would go permanently under water in twenty years.

SciAm probably expected a lot of people would agree with the key statement in their poll that the United Nations’ Intergovernmental Panel on Climate Change (IPCC) is “an effective group of government representatives and other experts.”

Hardly. As of this morning, only 16% of the 6655 respondents agreed.  84%—that is not a typo—described the IPCC as “a corrupt organization, prone to groupthink, with a political agenda.” 

The poll also asks “What should we do about climate change?” 69% say “nothing, we are powerless to stop it.” When asked about policy options, an astonishingly low 7% support cap-and-trade, which passed the U.S. House of Representatives in June, 2009, and cost approximately two dozen congressmen their seats.

The real killer is question “What is causing climate change?” For this one, multiple answers are allowed.  26% said greenhouse gases from human activity, 32% solar variation, and 78% “natural processes.” (In reality all three are causes of climate change.)

And finally, “How much would you be willing to pay to forestall the risk of catastrophic climate change?”  80% of the respondents said “nothing.”

Remember that this comes from what is hardly a random sample.  Scientific American is a reliably statist publication and therefore appeals to a readership that is skewed to the left of the political center.  This poll demonstrates that virtually everyone now acknowledges that the UN has corrupted climate science, that climate change is impossible to stop, and that futile attempts like cap-and-trade do nothing but waste money and burn political capital, things that Cato’s scholars have been saying for years.

Your Health Insurance, Designed by Lobbyists

Christopher Weaver of Kaiser Health News has an excellent article in today’s Washington Post on the various government agencies that will now be deciding what health insurance coverage you must purchase, and how many of those decisions will ultimately fall to lobbyists and politicians:

For years, an obscure federal task force sifted through medical literature on colonoscopies, prostate-cancer screening and fluoride treatments, ferreting out the best evidence for doctors to use in caring for their patients. But now its recommendations have financial implications, raising the stakes for patients, doctors and others in the health-care industry.

Under the new health-care overhaul law, health insurers will be required to pay fully for services that get an A or B recommendation from the U.S. Preventive Services Task Force…[which] puts the group in the cross hairs of lobbyists and disease advocates eager to see their top priorities – routine screening for Alzheimer’s disease, diabetes or HIV, for example – become covered services.

And it’s not just the USPSTF that will be deciding what coverage you must purchase:

[P]lans must also cover a set of standard vaccines recommended by the Advisory Committee on Immunization Practices, as well as screening practices for children that have been developed by the Health Resources and Services Administration in conjunction the American Academy of Pediatrics. Health plans will also be required to cover additional preventative care for women recommended under new guidelines that the Department of Health and Human Services is expected to issue by August 2011.

The chairman of the USPSTF says the task force will try “to stay true to the methods and the evidence… the science needs to come first.”  A noble sentiment, but as my colleague Peter Van Doren likes to say, “When politics and science conflict, politics wins.”  Witness how industry lobbyists have killed or neutered every single government agency that has ever dared to produce useful comparative-effectiveness research.  (You’re next, Patient-Centered Outcomes Research Institute!)

When government agencies are making non-scientific value judgments–e.g., are these studies reliable enough to merit an A or B recommendation? what should be the thresholds for an A or B recommendation? will the benefits of mandating this coverage outweigh the costs?–politics does even better.  Witness Sen. Barbara Mikulski (D-Md) overruling a USPSTF recommendation when she “inserted an amendment in the [new] health-care law to explicitly cover regular mammograms for women between 40 and 50. “

Speaking of value judgments, the one flaw in Weaver’s article is that it inadvertently conveys a value judgment as if it were fact.  He writes that the mandate to purchase coverage for preventive services is “good news for patients” and that 88 million Americans “will benefit.”  Whether the mandate is good news for patients depends on whether patients value the added coverage more than the additional premiums they must pay.  (The administration estimates that premiums for affected consumers will rise an average of 1.5 percent.  One insurer puts the average cost at 3-4 percent of premiums.  Naturally, some consumers will face above-average costs.)  Whether the benefits outweigh the costs is ultimately a subjective determination. (The best way to find out, as it happens, is to let consumers make the decision themselves.)

Ms. Weaver Goes to Washington

Today in Washington: actress Sigourney Weaver testifies before the  Subcommittee on Oceans, Atmosphere, Fisheries, and Coast Guard of the Senate Committee on Commerce, Science and Transportation Committee on the topic of ocean acidification. Because, you know, she played an environmental scientist in Avatar. It’s the best fit since Jane Fonda, Jessica Lange, and Sissy Spacek – all of whom had played farm women – testified on America’s agricultural crisis.

Congress doesn’t have time to vote on presidential nominations. It doesn’t bother engaging in serious oversight of presidential power and civil liberties abuses. It looks at the ceiling and whistles as the national debt approaches Greek levels. But members of Congress have time to listen to an actress discuss the topic of ocean acidification.

This seems like a topic for “Really!?! with Seth and Amy” on Saturday Night Live. Really, Senate Commerce Committee? You think Sigourney Weaver has important information that you need to know? Really? And you’re not just doing this to get yourselves on television? Really!?! And you think the most important thing members of Congress could be doing today is getting their pictures taken with Sigourney Weaver? Really!?!

Of course, this is not just a one-day thing for Sigourney Weaver. She also traveled this month to Brazil to try to stop the construction of a dam. Because who would know better than a Hollywood-Manhattan actress how to make tradeoffs between energy needs and environmental risks in Brazil?

Now let me just say that I’m not arguing that ocean acidification isn’t an important topic. And I’m not criticizing Avatar or its defense of property rights. I’m just questioning whether Sigourney Weaver, Sissy Spacek, Jeff Daniels, Nick Jonas, and the Backstreet Boys have the kind of expertise that Congress ought to draw on in deciding how to run my life. Or then again, maybe planning the economy and running other people’s lives is farce at best, and Congress should just hold hearings with Will Ferrell and John Cleese.

Who Wants to Make Sarah Palin the Leader of the Republican Party?

Could it be the Washington Post? Bannered across the top of the Post’s op-ed page today is a piece titled “Copenhagen’s political science,” titularly authored by Sarah Palin. I’m delighted to see the Post publishing an op-ed critical of the questionable science behind the Copenhagen conference and the demands for massive regulations to deal with “climate change.”

But Sarah Palin? Of all the experts and political leaders a great newspaper might call on for a critical look at the science behind global warming, Sarah Palin?

What’s even more interesting is that the Post also ran an op-ed by Palin in July. But during this entire year, the Post has not run any op-eds by such credible and accomplished Republicans as Gov. Mitch Daniels; former governors Mitt Romney or Gary Johnson; Sen. John Thune; or indeed former governor Mike Huckabee, who might be Palin’s chief rival for the social-conservative vote. You might almost think the Post wanted Palin to be seen as a leader of Republicans.

I should note that during the past year the Post has run one op-ed each from John McCain, Bobby Jindal, Newt Gingrich, and Tim Pawlenty. (And for people who don’t read well, I should note that when I call the people above “credible and accomplished,” that’s not an endorsement for any political office.) Still, it’s the rare political leader who gets two Post op-eds in six months, and rarer still the Post op-eds by ex-governors who can’t name a newspaper that they read.

NAEP Math Scores, NCLB, and the Federal Government

I’m surprised anyone was surprised by the recent flat-lining of scores on the NAEP 4th grade math test. The rate of improvement in NAEP scores has been declining since No Child Left Behind was passed, and the recent results are consistent with that trend.

But what really amazes me is that so many people think the solution is just to tweak NCLB! The unstated assumption here is that federal policy is a key determinant of educational achievement. That’s rubbish.

We’ve spent $1.8 trillion on hundreds of different federal education programs since 1965, and guess what: at the end of high school, test scores are flat in both reading and math since 1970, and have actually declined slightly in science. (Charted for your viewing pleasure here).

If we’ve proved anything in the past 40 years, it is that federal involvement in education is a staggering waste of money.

Meanwhile, education economists have spent the last several decades finding out what actually does work in education. They’ve compared different kinds of school systems and it turns out that parent-driven, competitive education markets consistently outperform state monopoly school systems like ours. I tabulated the results in a recent peer-reviewed paper and they favor education markets over monopolies by a margin of 15 to 1.

So policymakers who actually care about improving educational outcomes should be spending their time and resources enacting laws that will bring free and competitive education markets within reach of all families. And they should be ignoring the education technocrats who – like Soviet central planners – just want to keep spending other people’s money tweaking their fruitless five year plans.