April 28, 2009 2:30PM # You Just Can’t Say That By [Neal McCluskey](https://www.cato.org/people/neal-mccluskey) --- SHARE Let’s get one thing straight: As I’ve noted [on](https://www.cato.org/2008/06/19/what-fordham-cant-say-but-does-anyway/) [numerous](https://www.cato.org/author/neal-mccluskey/page/6/) [occasions](https://www.cato.org/pub_display.php?pub_id=8879), you can’t look just at National Assessment of Educational Progress (NAEP) results – especially only between two years – and attribute gains or losses to specific laws or programs. There are simply too many variables at play in education – federal laws, state laws, school choice, child nutrition, teacher quality, parents’ attitudes, the weather – to confidently assert that any one is responsible for changing scores. Indeed, it is possible that *nothing* government has done has had any effect, and every trend just reflects changing attitudes toward education among students themselves. And yet, some reporters identify something akin to a god variable anyway, as the Associated Press did in [its coverage](http://seattletimes.nwsource.com/html/politics/2009136678_apusreadingmathscores.html) of the [new NAEP long-term-trends report](http://nationsreportcard.gov/ltt_2008/): > The biggest gains came from low-achieving students. That is probably not an accident — the federal No Child Left Behind law and similar state laws have focused on improving the performance of minority and poor children, who struggle the most. Now, there are a lot of problems with this statement, including that several of the lowest-achieving percentiles by age and subject saw no statistically significant changes in scores between 2004 and 2008; many groups had periods of faster gains before NCLB (though we don’t even have clear before and after-NCLB data points); and NAEP offers no income-based score breakdowns, only the proxy of parents’ education – and that just for 13 and 17-year-olds in mathematics. But the biggest problem is that, all of these factual problems aside, *there is no way to ascribe score changes to specific laws or government policies*. The data just aren’t there. Fortunately, most of the coverage of the NAEP report has been pretty reasonable, including from the [*Washington Post* ](http://www.washingtonpost.com/wp-dyn/content/article/2009/04/28/AR2009042801244.html?hpid=moreheadlines)and *[New York Times](http://www.nytimes.com/2009/04/29/education/29scores.html?hp)*. But the AP reaches a lot of people, and that means many Americans are going to get “news” about the latest NAEP findings that is little more than unsupportable conjecture. ##### Related Tags [Education](https://www.cato.org/education), [Center for Educational Freedom](https://www.cato.org/center-educational-freedom) [![Creative Commons License](/build/cato_2020/images/creative-commons.svg)](http://creativecommons.org/licenses/by-nc-sa/4.0/) This work is licensed under a [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License](https://creativecommons.org/licenses/by-nc-sa/4.0/).