Topic: Education and Child Policy

OK Preschool Study Provides No Evidence of Lasting Benefits from Preschool

USA Today reports today on the findings of a preschool study, which concludes that Oklahoma’s government-run preschool gives a boost to the performance of all students in the short-term. Its news because the collective conclusions of previous studies overwhelmingly suggest that preschool boosts at-risk children in the short-term, but not children from middle and upper-income families.

An ambitious public pre-kindergarten program in Oklahoma boosts kids’ skills dramatically, a long-awaited study finds, for the first time offering across-the-board evidence that universal preschool, open to all children, benefits both low-income and middle-class kids…

More than any other state, Oklahoma has pushed for universal pre-kindergarten, with the USA’s highest enrollment rate.

There’s just one tiny problem. Oklahoma’s achievement scores on National Assessment of Educational Progress (NAEP, AKA “the nation’s report-card”) suggest that the state’s universal preschool program is at best ineffective and at worst harmful to student achievement.

Oklahoma, “where state-funded pre-kindergarten has been in place for 18 years — and offered universally for nearly a decade,” has slipped below the national average on math and reading scores for both the 4th and 8th grades since it began expanding government pre-K in the 1990’s.

Oklahoma slipped from one point above the national average in 4th grade math in 1992 to two points behind in 2007. They slipped further behind in 8th grade math, from one point ahead to five points behind the national average. In reading the stories the same; 8th grade scores slipped from four points ahead in 1998 to one point behind. And Oklahoma’s 4th grade reading scores plummeted during the 1990’s at the very same time the state was aggressively expanding preschool access, increasing attendance, and building a system that the National Institute for Early Education Research (NIEER) rates 9 out of 10 on quality.

There is little evidence in the research that these kinds of preschool programs impart lasting gains to low-income students and no evidence that they benefit middle-class kids. The real-world evidence demonstrates that the test scores of children in Oklahoma have eroded significantly, as have our nation’s performance on international tests, at the same time that preschool programs have massively expanded and the quality of those programs has supposedly improved.

Tax Credits We Don’t Need, Tax Credits We Do … Ohio #1

Here is the first of what might become a series depending on the fan mail …

We have tax credits for all manner of things at the federal, state, and local levels that we don’t need and shouldn’t have tax credits for … like hybrid cars, movie production, and lemonade stands (the last one isn’t real as far as I know, but I wouldn’t rule it out).

One of the more popular tax credits is for saving old buildings that some people don’t want torn down but don’t care enough about to save with their own collective money. So they subsidize the renovation with credits. The Plain Dealer editorializes in support:

A state historic tax credit that, shortsightedly, was drastically scaled back this spring has now been expanded, thanks in large part to advocacy by Russell Township Rep. Matt Dolan - a leading GOP candidate to become the next Ohio House speaker - and support from Lt. Gov. Lee Fisher.

The program was revived when Gov. Ted Strickland signed a $1.6 billion economic stimulus bill. That will allocate $120 million in tax credits for developers to preserve and renovate historic properties, most of which are in urban areas.

What most states don’t have are education tax credits – the one and only tax credit that makes fiscal sense because it really does save taxpayers’ money and the only tax credit that actually decreases market distortion rather than increasing it.

But I have a question for Ohio’s politicians; if it’s good to encourage developers to invest in building preservation, why isn’t it good to encourage all taxpayers to invest in education? Are developers and old buildings more important than a child’s future?

Will the Plain Dealer, Rep. Matt Dolan, Lt. Gov. Lee Fisher, and Gov. Ted Strickland come out in support of at least $120 million in tax credits for educating Ohio’s children? If not why not?

Inquiring minds want to know …

So Much, Yet So Little

Depending on where you live, you might have seen a story in your local paper on a new report finding that test scores have improved under the No Child Left Behind Act, implying — but not outright saying — that NCLB is working.

So why would getting this news depend on where you live? Because the report looked primarily at state tests, and only about 28 states had sufficiently consistent data to do meaningful score analyses. The other 22 had changed their testing in so substantial a way since NCLB’s passage that not even three years of consistent results could be strung together. Which is the biggest non-finding finding of the report: NCLB has instigated so much test engineering — often to make assessments easier — that nearly half of all states have no useful long-term data. And don’t automatically assume that the other 28 haven’t changed things: the report itself notes that they could have made questions easier or harder, changed relative weights of questions, and made other, more subtle changes to their tests.

There are other problems with the study that make it impossible to credit score increases to NCLB, most of which the report is upfront about but news stories rarely feature: There’s no “control group” unaffected by NCLB against whom to compare students “treated” by the law, no ability to tease out the effects of NCLB versus other reforms, etc. The most obvious problem, though, is that in large part because of NCLB, lots of states have testing systems incapable of providing the consistent, long-term results that federal policymakers promised the law itself would deliver.

Conflicting Data? What Conflicting Data?

The public school advocacy group Center on Education Policy released a new report today, titled “Has Student Achievement Increased Since 2002?” Its answer is “yes,” based on relatively worthless high-stakes state-level testing data and on the more esteemed National Assessment of Educational Progress (NAEP). For reasons known only to the report’s authors, they make no use of the available U.S. trend data from either the PISA or the PIRLS international tests (though the CEP study mentions PISA results for a single point in time, it ignores the changes in that test’s scores over time.)

As it happens, U.S. scores have declined on both PISA and PIRLS in every subject and at both grades tested since they were first administered in 2000/2001. In the PISA mathematics and science tests, the declines are large enough to be statistically significant, that is: we can be confident (and disappointed) that they reveal real deterioration in U.S. student performance. In mathematics, our score has dropped from 493 to 474, causing us to slip from 18th out of 27 participating countries down to 25th out of 30 countries. In science, our score fell from 499 to 489, dropping us from 14th out of 27 countries to 21st out of 30 countries.

It is reckless and misleading to form judgments about trends in U.S. student performance without taking into account the declines on these respected international tests. And, as Neal McCluskey and I pointed out last year, the improving trends that exist on some NAEP tests predate the passage of the No Child Left Behind act, and have in some cases actually slowed since that law’s passage.

It is this rather discouraging reality that should guide policymakers in the coming year, as they debate the future of NCLB.

Fordham Follow Up

Yesterday, I wrote about a new Thomas B. Fordham Institute report on high-achieving students. I focused mainly on a bit of hyperbole issued by Fordham president Chester Finn and vice president Michael Petrilli concerning the “plainly” positive effect of government “standards and accountability” on low-achieving kids. Today, I just want to add a quick, seemingly obvious observation — but one barely hinted at in the report — concerning survey results that show teachers think high achievers in their schools get short shrift.

“Teachers say that while the public schools muster serious effort to improve the academic achievement of struggling students, their resources rarely converge on the needs of high achievers,” report Stave Farkas and Anne Duffett, who handled the survey section of Fordham’s report.

The problem is — you’ve got it! — one-size can’t fit all! A single system of public schools — especially one ever-more centralized with crude government accountability mandates like the No Child Left Behind Act — can only focus on one or two things at a time, and high achievers aren’t it.

So what’s the obvious solution? School choice!

In a market, consumers purchase things according to their individual preferences, needs, abilities, etc., and producers tailor products accordingly. I need a small, fuel-efficient car, I get a Corolla. You want a big beast to haul your soccer balls, you get a Suburban. I want dinner that’s also entertainment, I go to Benihana. You want buttered lobster bites, you head to Long John Silver’s.

When producers and consumers are free, diverse preferences that seem almost infinite are met by equally diverse providers, and everyone is better off. With school choice, gifted kids could go to schools that specialize not only in nurturing gifted children, but specific talents like artistic ability or scientific acumen. On the flip side, students with specific learning disabilities could go to schools that focus on their problems. In contrast, when everyone gets what government tells them they’re going to get, well, you know what happens.

Sadly, the only choices mentioned in Fordham’s report are district-run magnet schools and programs, but since the survey was of public school teachers, that’s understandable.  What’s not understandable is why in their NRO piece yesterday, Finn and Petrilli, while lionizing government “accountability” for supposedly helping low-achieving students, didn’t point to the huge promise of markets to educate each and every student.

What Fordham Can’t Say, But Does Anyway

Yesterday, the Thomas B. Fordham Institute released a report suggesting that the No Child Left Behind Act has encouraged schools to focus on the lowest-performing students and neglect the highest-performing. This is not an unreasonable hypothesis: National Assessment of Educational Progress data suggest it could be true, though the results are mixed and Tom Loveless, the author of the report’s NAEP analysis (the report also includes some interesting survey results), makes it clear that it is impossible to say what, if any, test-score changes have been caused by NCLB.

Unfortunately, the spin put on the “good news” in the report by Fordham president Chester Finn and vice president Mike Petrilli is not nearly as measured as Loveless’ caveat about NCLB. On National Review Online today, Finn and Petrilli write with total certainty that government-driven “standards and accountability” regimes have produced gains for low-performers.

“NCLB and state-level efforts to impose standards and accountability on the schools are plainly boosting the kids who need it most — surely a good thing,” they pronounce.

Rising achievement surely is a good thing. That government standards and accountability produced it, however, is far from sure.

First, compare the period that contains NCLB, which was passed in 2002, to score changes in the period preceding it. In reading, the lowest 10 percent of 4th grade performers saw a much bigger increase in scores immediately before 2002 than after, and 8th graders saw their scores drop under NCLB. In math, we have to start with 2003, the earliest testing year within the NCLB timeframe. Again, for the lowest performers, in both 4th and 8th grades scores increased faster in the period right before NCLB — 2000 to 2003 — than after.

Loveless notes in the report that it is impossible to be sure what effect NCLB had on math in the 2000 to 2003 period — where the fastest gains are seen — since NCLB was passed in 2002. He’s right. However, in light of long delays in issuing NCLB regulations, and the unlikelihood of a huge jump in just one year of NCLB, it is more reasonable not to ascribe improvements to the law than to give it credit. More importantly, one definitely cannot say, as Finn and Petrilli nonetheless do, that the law “plainly” has something to do with rising low-achiever scores.

To be fair, Finn and Petrilli say NCLB and “state-level efforts” — not just NCLB — boosted those scores. On what basis do they split credit?

In his analysis, Loveless examined states’ NAEP score changes for the highest and lowest performers, controlling for whether or not states had their own standards and accountability regimes before NCLB. Unfortunately, the report doesn’t list which are considered “accountability” and which “non-accountability” states, so it is impossible to search for other common characteristics — charter schools, private-school-choice programs, increasingly affluent populations, new curricula — that could have driven states’ performances. Even more damaging to Finn and Petrilli’s pronouncement, the data the report does make available simple cannot support their all-too-firm-sounding conclusion.

For one thing, for the four subject-grade combinations presented, only between 34 and 37 states are analyzed, leaving out one-third of the country. More important, while in three of the four subject-grade combinations the lowest performers in states with accountability regimes did see greater score increases than low perfomers in states without them, when you only have four comparisons you simply cannot declare uncontestable victory, much less when only three of the comparisons support your conclusion. Change one, and you’ve got a coin flip. Unfortunately, that didn’t stop Petrilli and Finn.

In the end, Fordham’s new report doesn’t tell us anything definitive about the effect of NCLB or any other standards and accountability regimes. It offers some reason to believe that NCLB might help low scorers and leave high scorers behind — and it’s well worth reading just for that — but it provides nothing close to proof. It also suggests that standards and accountability regimes might help raise low-performers’ scores, but again has far too many holes and far too little information to support what Finn and Petrilli declare: that government-imposed standards and accountability “plainly” help low-achieving kids.

Suburban Opposition to Choice and the Money Misperception

Andrew Coulson has a great response to a recent “Best of the Web” column by WSJ’s James Taranto, which notes that there is widespread and self-interested opposition to vouchers from wealthier parents and homeowners.

I just wanted to add a bit about two things Taranto suggests are a major concern limiting school choice success; property values and taxes. He’s wrong on property values, but correct about taxes.

Coulson notes that the property value effects of choice are not as predictable as many political elites think, and that might help explain one interesting finding from my doctoral research.

In a large-scale survey of close to 2,900 respondents, I found that property value concerns were a negligible consideration in regard to school choice. In fact, around 40 percent of respondents think that property values will increase with school choice. Most of the rest think choice would have no impact at all on property values. And even high-income respondents without school-aged children believe, by 30 percent to 16 percent, that the adoption of school choice policy will increase property values in their area. 

Property values, in other words, do not seem to be an important drag on support for school choice. Coulson points to what does seem to be the major concern for higher-income suburbanites; cost.

Most people think that school choice will increase academic achievement and have other beneficial effects. But most people also believe, incorrectly, that choice will substantially increase costs. And why wouldn’t they? What new government program promising substantial improvements in anything ever cost taxpayers less?

Regression analyses reveal that cost concerns are the biggest drag on support. It should therefore come as no surprise that exposing respondents to an argument for school choice that emphasizes the cost savings was the most effective in increasing support for school choice.

If we want to make inroads with those who are skeptical of school choice, we need to do more to educate them on the fiscal benefits of choice.