Tag: national assessment of educational progress

Arne Duncan’s Chicago Schools

The Washington Post reports on what new data reveal about the Chicago public schools run for the past seven years by Arne Duncan, now President Obama’s secretary of education:

This month, the mathematics report card was delivered: Chicago trailed several cities in performance and progress made over six years.

Miami, Houston and New York had higher scores than Chicago on the National Assessment of Educational Progress. Boston, San Diego and Atlanta had bigger gains. Even fourth-graders in the much-maligned D.C. schools improved nearly twice as much since 2003.

As I’ve said before, what always struck me about Obama’s appointment of Duncan to run the nation’s schools – and he is actually moving to do just that, more so than any previous federal administration – is that Arne Duncan ran the Chicago schools for seven years, and in that time he didn’t manage to produce a single school that the Obamas chose to send their own children to. Valerie Schwartz of the Post reminds us that Duncan is not the first Cabinet secretary to be appointed on the basis of great results in a previous job, that then turned out to be not so great.

Of course, you could have read much of the data about Duncan’s results right here at Cato @ Liberty back in July.

Another Education Road Sign Screaming “Stop!”

This morning the National Center for Education Statistics released a new report, Mapping State Proficiency Standards Onto NAEP Scores: 2005-2007.  What the results make clear (for about the billionth time) is that government control of education has put us on a road straight to failure. Still, many of those who insist on living in denial about constant government failure in education will yet again refuse to acknowledge reality, and will actually point to this report as a reason to go down many more miles of bad road.

According to the report, almost no state has set its “proficiency” levels on par with those of the National Assessment of Educational Progress (NAEP), the so-called “Nation’s Report Card.” (Recall that under No Child Left Behind all children are supposed to be “proficient” in reading and math by 2014.) Most, in fact, have set “proficiency” at or below NAEP’s “basic” level. Moreover, while some states that changed their standards between 2005 and 2007 appeared to make them a bit tougher, most did the opposite. Indeed, in eighth grade all seven states that changed their reading assessments lowered their expectations, as did nine of the twelve states that changed their math assessments.

Many education wonks will almost certainly argue that these results demonstrate clearly why we need national curricular standards, such as those being drafted by the Common Core State Standards Initiative. If there were a national definition of “proficiency,” they’ll argue, states couldn’t call donkeys stallions. But not only does the existence of this new report refute their most basic assumption – obviously, we already have a national metric – the report once again screams what we already know:  Politicians and bureaucrats will always do what’s in their best interest – keep standards low and easy to meet – and will do so as long as politics, not parental choice, is how educators are supposed to be held accountable. National standards would only make this root problem worse, centralizing poisonous political control and taking influence even further from the people the schools are supposed to serve. 

Rather than continuing to drive headlong toward national standards – the ultimate destination of the pothole ridden, deadly, government schooling road – we need to exit right now. We need to take education power away from government and give it to parents. Only if we do that will we end hopeless political control of schooling and get on a highway that actually takes us toward excellent education.

Federal Education Results Prove the Framers Right

Yesterday, I offered the Fordham Foundation’s Andy Smarick an answer to a burning question: What is the proper federal role in education? It was a question prompted by repeatedly mixed signals coming from U.S. Secretary of Education Arne Duncan about whether Washington will be a tough guy, coddler, or something in between when it comes to dealing with states and school districts.  And what was my answer? The proper federal role is no role, because the Constitution gives the feds no authority over American education.

Not surprisingly, Smarick isn’t going for that. Unfortunately, his reasoning confirms my suspicions: Rather than offering a defense based even slightly on what the Constitution says, Smarick essentially asserts that the supreme law of the land is irrelevant because it would lead to tough reforms and, I infer, the elimination of some federal efforts he might like.

While acknowledging that mine is a “defensible argument,” Smarick writes that he disagrees with it because it “would presumably require immediately getting rid of IDEA, Title I, IES, NAEP, and much more.” He goes on to assert that I might “argue that doing so is necessary and proper because it’s the only path that squares with our founding document, but policy-wise it is certainly implausible any time soon.” Not far after that, Smarick pushes my argument aside and addresses a question to “those who believe that it’s within the federal government’s authority to do something in the realm of schools.”

OK. Let’s play on Smarick’s grounds. Let’s ignore what the Constitution says and see what, realistically, we could expect to do about federal intervention in education, as well as what we can realistically expect from continued federal involvement.

First off, I fully admit that getting Washington back within constitutional bounds will be tough. That said, I mapped out a path for doing so in the last chapter of Feds In The Classroom, a path that doesn’t, unlike what Smarick suggests, require immediate cessation of all federal education activities. Washington obviously couldn’t be pulled completely out of the schools overnight.

Perhaps more to Smarick’s point, cutting the feds back down to size has hardly been a legislatively dead issue. Indeed, as recently as 2007 two pieces of legislation that would have considerably withdrawn federal tentacles from education – the A-PLUS and LEARN acts – were introduced in Congress. They weren’t enacted, but they show that getting the feds out of education is hardly a pipe dream. And with tea parties, the summer of townhall discontent, and other recent signs of revolt against big government, it’s hardly out of the question that people will eventually demand that the feds get out of their schools.

Of course, there is the other side of the realism argument: How realistic is it to think that the federal government can be made into a force for good in education? It certainly hasn’t been one so far. Just look at the following chart plotting federal education spending against achievement, a chart that should be very familiar by now.

Education Spending

Notice anything? Of course! The federal government has spent monstrous sums on education without any corresponding improvement in outcomes!

Frankly, it’s no mystery why: Politicians, as self-interested people, care first and foremost about the next election, not long-term education outcomes. They care about what will score them immediate political points. That’s why federal politicians have thrown ever-more money at Title I without any meaningful sign it makes a difference. That’s why No Child Left Behind imposed rules that made Washington politicians look tough on bad schools while really just pushing more dough at educrats and giving states umpteen ways to avoid actual improvement. That’s why Arne Duncan vacillates between baddy and buddy at the drop of a headline. And that basic reality – as well as the reality that the people employed by the public schools will always have the greatest motivation and ability to influence government-schooling policies – is why it is delusional to expect different results from federal education interventions than what we’ve gotten for decades.

OK. But what about a law like the Individuals with Disabilities Education Act (IDEA)? Hasn’t it helped millions of disabled kids who would otherwise have been neglected by states and local school districts?

For one thing, it is constitutional and totally appropriate under the 14th Amendment for the federal government to ensure that states don’t discriminate against disabled children in provision of education. IDEA, however, does much more than that, spending billions of federal dollars, promoting over-identification of “disabilities,” and creating a hostile, “lawyers playground” of onerous, Byzantine rules and regulations, all without any proof that the law ultimately does more good than harm. And again, this should be no surprise, because federal politicians care most about wearing how much they “care” on their reelection-seeking sleeves, no matter how negative the ultimate consequences may be.

Alright-y then. How about the National Assessment of Educational Progress (NAEP)? Isn’t it an invaluable source of national performance data?

NAEP results are used in the above chart, so obviously I have found NAEP of some value.  But does its usefulness justify ignoring the Constitution? Absolutely not. For one thing, instead of NAEP we could use extant, non-federal tests such as the SAT, ACT, PSAT, Stanford 9, Terra Nova, and many other assessments to gauge how students are doing. And as useful as NAEP may be, it sits perilously close to being as worthless as everything else that Washington has done in education. All that has kept it from being hopelessly politicized is that there is no money attached to how states and local districts do on it. And as Smarick’s boss at Fordham, Chester Finn, testified in 2000, even with that protection NAEP and other supposedly netural federal education undertakings are under constant threat of political subversion:

Unfortunately, the past decade has also shown how vulnerable these activities are to all manner of interference, manipulation, political agendas, incompetence and simple mischief. It turns out that they are nowhere near to being adequately immunized against Washington’s three great plagues:

• the pressing political agendas and evanescent policy passions of elected officials (in both executive and legislative branches)and their appointees and aides,

• the depredations and incursions of self-serving interest groups and lobbyists (of which no field has more than education), and

• plain old bureaucratic bungling and incompetence.

Based on all of this evidence, it is clear that the only realistic avenue for getting rational federal education policy is, in fact, to follow the Constitution and have no federal education policy. In other words, the very realistic Framers of the Constitution were absolutely right not to give the federal government any authority over education, and it is time, right now, for us to stop ignoring them. Doing anything else will only ensure continued, bankrupting failure.

All That NAEP Tells Us Is Things Ain’t Good

Yesterday, another round of scores on the National Assessment of Educational Progress – the so-called “Nation’s Report Card” – came out. They revealed flattened 4th-grade math achivement between 2007 and 2009, and a two point (out of 500) increase in 8th grade.

So what do these bits of data portend? Ask the experts:

“The trend is flat; it’s a plateau. Scores are not going anywhere, at least nowhere important,” said Chester Finn, president of the Thomas B. Fordham Institute, according to the New York Times. “That means that eight years after enactment of No Child Left Behind, the problems it set out to solve are not being solved, and now we’re five years from the deadline and we’re still far, far from the goal.”

Next, former National Center for Education Statistics commissioner Mark Schneider concluded that “either the standards movement has played out, or the No Child law failed to build on its momentum. Whatever momentum we had, however, is gone.”

And then there’s Michigan State University professor William Schmidt, a leading national-standards proponent, who opined in the Baltimore Sun that “there is a hardly any change. There is hardly any difference. How could we as a nation let that happen?” His solution to the problem: National standards, of course.

So what do I think about all this? As a long-time critic of NCLB, I am glad to see people seizing on the latest results and declaring the law a failure. It helps to advance my goal of ending the greatest federal education intervention to date, and I think NCLB supporters kind of deserve these attacks on their law. They have repeatedly given NCLB credit for positive things the evidence couldn’t come close to supporting, and it’s nice to see them on the defensive after all their overreaching.

That said, just as previous NAEP results couldn’t prove that NCLB was working, the latest NAEP scores don’t prove that it is not. We simply don’t have sufficient information about the myriad other variables affecting education to do that.

Which leads me to a much bigger problem: People using ambiguous  NAEP scores to push their favorite reforms. Some “standards and accountability” proponents, for instance,  argue that achievement improvements came as a result of states implementing standards-and-testing regimes during the 1990s. And William Schmidt suggests that the latest NAEP results demonstrate a need for national standards.

Now, NAEP simply cannot be used in any reasonable way to justify the national-standards assertion. We’ve never had national standards, so we obviously can’t measure their outcomes with NAEP. We can, however, attempt to use NAEP to assess the assertion that the push for state standards and testing in the 1990s drove real improvement. We can attempt, that is, but any conclusions will be riddled with problems.

Let’s start with the NAEP exam from which yesterday’s results came, the “main” mathematics exam. After that, we’ll look at some “long-term” NAEP results.

Take a look at the chart below. It is the 4th-grade trend line for the main math NAEP, with vertical lines separating what I’ll call the pre-accountability period (1990-1996), the state-accountability period (1996-2003), and the NCLB period (2003-2009). The numbers below each period are the per-year changes in scores for the periods above them.

RegMath41996

What do the numbers show us? At first blush, the message seems to be that scores were improving slightly faster in the pre-accountability period than the vaunted state-accountability period, and appreciably faster than under NCLB. But this breakdown may beg more questions than it answers.

At what year, for instance, should you peg the start of the state-accountability period? I chose 1996 because by then, according to a count by Hanushek and Raymond (Table 1), twelve states – a pretty large number – had some sort of accountability mechanism in place. I could, however, have chosen 1992, because by 1993 three states had such mechanisms and the accountability movement could certainly be said to have been underway. Similarly, NCLB was enacted in 2002 – is it right to start the NCLB period in 2003? Obviously I couldn’t start the period in 2002 because there is no data for that year, but why not 2005? After all, though enacted in 2002, NCLB took a few years to be fully implemented.

Let’s make an adjustment. I’m going to keep the start of the NCLB era at 2003 because that’s pretty close to the enactment year – though it could very well produce misleading results – but will move the end of the pre-accountability period, and hence the start of the state-accountability period, to 1992.
RegMath41992

Now what should we conclude? Again, it appears that the pre-accountability period had the best results, but this time by a much bigger margin. That said, that period included only two years – hardly sufficient data to identify a trend. Also, the NCLB period fared better than previously against the state-accountability years.

Of course, the main NAEP gives us data for less than two decades. So what does the long-term NAEP show for 9-year-olds (roughly 4th graders) in math?

Here’s a long-term trend chart, which like the main NAEP charts above is broken into periods with score-change-per-year noted below. I’ve broken it into the period before the 1983 publication of the landmark A Nation at Risk report, which scared people silly about the schools; the post-ANARbut pre-state-accountability period; the state-accountability period; and the NCLB period.
longterm96

What do these results show?

In contrast to the main NAEP scores, the greatest improvement on the long-term test occurred during the state accountability period, and the second greatest under NCLB. But again, this raises more questions than it answers: Why the difference between the main and long-term results? When is best to start each period? Does it make any sense to start a period with A Nation at Risk? Should the NCLB period start in 1999, well before the law was enacted, or in 2004, two years after it’s passage? And the questions go on.

Like we did with the main NAEP results, let’s once again look at a different start date, 1992, for the state-accountability period, this time on the long-term exam.

longterm92

Once again we get a whole different story. Now it is the pre-accountability period, not the state accountability period, that shows the best outcomes. And so the ambiguity continues…

All of this, of course, goes to show that NAEP results cannot be used with any confidence to conclude that any particular reform that occurred within the NAEP time span worked better or worse than any other reform within that span.

That said, there is one thing that we canuse NAEP to demonstrate very powerfully: As Andrew Coulson’s chart below vividly illustrates, if moving the achievement needle as measured by NAEP is the goal of education spending, then we have really been getting robbed! Moreover, to the extent that standards-based reforms have been a major national phenomenon since the 1990s, it is impossible to conclude that they have done very much good. Indeed, if we are to conclude anything, it is that it is time to focus on reforms that are completely different from the top-down “solutions” that have given us so little, and taken so much.

andrew-coulson-cato-education-spending

Research Shows $100 Billion Ed. Stimulus Likely Hurting Economy

Tomorrow morning, the president’s Council of Economic Advisers will release a report assessing the short and long-term effects of the stimulus bill on the U.S. economy. As with previous iterations, this report will attempt to forecast overall effects of the stimulus across its many different components and the different economic sectors it targets. In doing so, it ignores the clearest research findings available pertaining to a key portion of the stimulus: k-12 education.

The president has committed $100 billion in new money to the nation’s public school systems, and required that states accepting the funds promise not to reduce their own k-12 spending. The official argument for this measure is that higher school spending will accelerate U.S. economic growth. But a July 2008 study in the Journal of Policy Sciences finds that, to the authors’ own surprise, higher spending on public schooling is associated with lower subsequent economic growth. Spending more on public schools hurts the U.S. economy.

How is that possible? There is little debate in academic circles that raising human capital – improving the skills and knowledge of workers – boosts productivity. So an obvious interpretation of the JPS study is that raising public school spending must not increase human capital. While this possibility surprised study authors Norman Baldwin and Stephen Borrelli, it is consistent with the data on U.S. educational productivity over the past two generations.

Since 1970, inflation adjusted public school spending has more than doubled. Over the same period, achievement of students at the end of high school has stagnated according to the Department of Education’s own long term National Assessment of Educational Progress. Meanwhile, the high school graduation rate has declined by 4 or 5%, according to Nobel laureate economist James Heckman. So the only thing higher public school spending has accomplished is to raise taxes by about $300 billion annually, without improving outcomes.

The fact that more schooling without more learning is not a recipe for economic growth is confirmed by the independent empirical work of economists Eric Hanushek and Ludger Woessmann. Their key finding is that academic achievement, not schooling per se, is what matters to economic growth.

Based on this body of research, the president’s decision to pump $100 billion into existing public school systems is likely slowing the U.S. economic recovery.

You Just Can’t Say That

Let’s get one thing straight: As I’ve noted on numerous occasions, you can’t look just at National Assessment of Educational Progress (NAEP) results – especially only between two years – and attribute gains or losses to specific laws or programs. There are simply too many variables at play in education – federal laws, state laws, school choice, child nutrition, teacher quality, parents’ attitudes, the weather – to confidently assert that any one is responsible for changing scores. Indeed, it is possible that nothing government has done has had any effect, and every trend just reflects changing attitudes toward education among students themselves.

And yet, some reporters identify something akin to a god variable anyway, as the Associated Press did in its coverage of the new NAEP long-term-trends report:

The biggest gains came from low-achieving students. That is probably not an accident — the federal No Child Left Behind law and similar state laws have focused on improving the performance of minority and poor children, who struggle the most.

Now, there are a lot of problems with this statement, including that several of the lowest-achieving percentiles by age and subject saw no statistically significant changes in scores between 2004 and 2008; many groups had periods of faster gains before NCLB (though we don’t even have clear before and after-NCLB data points); and NAEP offers no income-based score breakdowns, only the proxy of parents’ education – and that just for 13 and 17-year-olds in mathematics. But the biggest problem is that, all of these factual problems aside, there is no way to ascribe score changes to specific laws or government policies. The data just aren’t there.

Fortunately, most of the coverage of the NAEP report has been pretty reasonable, including from the Washington Post and New York Times. But the AP reaches a lot of people, and that means many Americans are going to get “news” about the latest NAEP findings that is little more than unsupportable conjecture.

Educational Productivity Has Collapsed — NAEP

The latest Long Term Trends results of the National Assessment of Educational Progress are out. They reveal a productivity collapse unparalleled in any other sector of the economy.

At the end of high school, students perform no better today than they did nearly 40 years ago, and yet we spend more than twice as much per pupil in real, inflation-adjusted terms. I can’t think of any other service that has gotten worse during my lifetime. Our school system has failed alone.

While the stagnation in overall achievement masks a 3 to 5 percent gain in the achievement of African American 17-year-olds since 1970, the scores for whites at the end of high school are virtually unchanged.

Anyone who points to the slightly higher scores in the early grades as cause for celebration is missing the point. What parents care about is that their children are well prepared for higher education and future careers at the end of their secondary education. The fact that scores have risen somewhat in the early grades means little since those gains evaporate for the vast majority of students by the time they graduate.

Update: The Associated Press story is now out on the Long Term Trends NAEP results… and it doesn’t mention the long term trends. The story only reports changes in achievement over the most recent 4 year interval of a test whose raison d’être is to reach back to the early 1970s. I wonder why…. Fortunately, the Detroit Free Press does a better job.