Tag: test scores

Rick Perry, Arne Duncan, and Michael Jackson

By
Andrew J. Coulson

To my astonishment, Arne Duncan went after Republican presidential candidate Rick Perry yesterday on the grounds that Perry hasn’t done enough to improve the schools under his jurisdiction. According to Bloomberg News, Duncan said public schools have “really struggled” under Perry and that “Far too few of [the state’s] high school graduates are actually prepared to go on to college.”

I was never a huge Michael Jackson fan, but for some reason his “Man in the Mirror” track just popped into my head as I read this. You see, once upon a time, Arne Duncan was “CEO” of the Chicago Public Schools. During and for some time after his tenure, he was celebrated as having presided over “The Chicago Miracle,” in which local students’ test results had improved dramatically. That fact turns out to have been fake, but accurate. The state test results did improve, but not because students had learned more; they appear to have improved because the tests were dumbed-down.

When this charge was first leveled, I decided to look into it myself, and found that it was indeed justified. There was no “Chicago Miracle.” Arne Duncan ascended to the throne of U.S. secretary of education, at least in part, on a myth. The academic achievement of the children under his care stagnated at or slightly below the level of students in other large central cities during his time at the helm. Seems an opportune occasion for someone to “start with the man in the mirror, asking him to change his ways.”

Topics: 
Education and Child Policy

Fordham Institute Reviews ‘The Other Lottery’

By
Andrew J. Coulson

Gerilyn Slicker, of the Fordham Institute, offers a brief review of my study of charter school philanthropy in the latest “Education Gadfly” mailing, including the following observation:

Note, though, that this analysis is not without fault. The report doesn’t break down spending by pupil (only reporting aggregate grant-giving), nor does it account for student growth over time or for how long the charter networks have been operational.

All three of these concerns are worth raising, and the first two of them were actually addressed in the paper itself. The aggregate vs. per-pupil grant funding question is discussed in endnote 15:

Note that total grant funding, rather than grant funding per pupil, is the correct measure. That is because enrollment is endogenous—it is a product, in part, of earlier grant funding. So, controlling for enrollment (which dividing by enrollment would do) would control away some of the very characteristics we are trying to measure: the charter network’s ability to attract funding.

Student growth over time, as noted on page 5, cannot be measured using the California Standards Tests, because it reports results as averages of subgroups of students at the classroom level, not as individual student scores. And since the CST is the only source that has broad-based performance data for all charter and traditional public schools in the state, it is the only dataset that can be used to measure the performance of all California charter school networks. Fortunately, good controls for both student factors and school-wide peer effects are available, and the study’s results are consistent with earlier research, where it overlaps with that research.

The final concern, network age, is not one that I directly addressed in the study. There are a couple of reasons to expect it would not have much of an impact on the findings, however. First, a cursory look at the age of some of the top networks shows no particular pattern. American Indian and KIPP are both a decade old, and rank #1 and #7, respectively, out of 68 networks. Oakland Charter Academies and Rocketship are just a couple of years old, and rank #2 and #4, respectively. Similarly, some of low performing networks are brand new, while others, like GreenDot (ranked 42nd), are also over a decade old.

Second, in Appendix E, I show that network size and network academic performance are not significantly linked to one another. And since network age and network enrollment are likely to be strongly positively correlated with one another, it would be surprising if network age were correlated with performance when enrollment is not. That said, I’d probably include network age as a control in future, if I repeat the study, just to be on the safe side.

Topics: 
Education and Child Policy

Ranking the Charter School Networks

By
Andrew J. Coulson

Much of the response to the study I released last week has focused on the relative academic performance rankings of California’s charter school networks. That wasn’t the point of the study, which focuses on whether or not philanthropy + charter schooling can replace venture capital and competitive markets as a mechanism for scaling-up the best education services. Rather than try to fight the tide, I thought I’d just share the relevant rankings in an easy-to-link form, and once the debate about them dies down we can return to the larger policy point.

With that in mind, the first table below lists the top 15 charter school networks in terms of performance on the California Standards Tests, adjusted for student factors and peer effects. For comparison, two non-charter schools are included: the academically selective elite public prep schools Gretchen Whitney and Lowell–both of which feature in most lists of the top public schools in the country. There are 68 networks with the necessary data, but the lowest grant rank is 61 because eight of the networks received no philanthropic funding at all.

Next is a list of the charter networks that philanthropists have invested-in most heavily, with a view to replicating their models. Notice the minimal overlap? I repeat this comparison in the study with Advanced Placement test performance, and find the same pattern (it’s just slightly worse).

Every one of the above networks received substantially more grant funding individually than the top three highest achieving networks… combined.

Topics: 
Education and Child Policy

Reply to Samuelson: It Is an Engineering Problem

By
Andrew J. Coulson

In today’s Washington Post, Robert Samuelson argues that the performance of U.S. public schools is at least adequate, and that the relatively low achievement of black and Hispanic students is to be attributed to history and culture rather than to our education system. These claims are not new, and I might well have ignored them if he hadn’t got my Irish up with the off-hand comment that “what we face is not an engineering problem.” (More on that in a second.)

First, let’s dispatch the claim that public schooling is off the hook for the poor performance of low-income minority children. I’m currently undertaking a statistical study of the performance of 78 separate charter school networks in California, relative to one another and to the state’s traditional public schools. To foreshadow the results, the performance differences within socioeconomic groups are enormous even after controlling for school-wide peer effects. Among low-income Hispanic students, across grades, schools and subjects, average scores at two of the top charter networks (American Indian Public Schools and Oakland Charter Academies) are roughly 4 standard deviations above the statewide traditional public school mean. Quatre. Quattro. FOUR.

To put that in perspective, effect sizes in social science research are normally evaluated based on Jacob Cohen’s rule of thumb that 0.2 standard deviations is “small”, 0.5 is “moderate”, and anything bigger than 0.8 is “large.” To put it further in perspective, the low-income Hispanic effect sizes of two of California’s most elite and academically selective public schools are closer to 2 S.D. So the top charter networks, which accept every student who applies, massively outperform elite public schools that actively select their students based on prior test scores. Consistently. Across grades and subjects. [Note that there’s also wide variation in performance among charter school networks, with many performing below the mean of traditional public schools. Further details when the paper is published in a few months].

So, no, public schooling is not off the hook. We know it is possible to dramatically raise the achievement of low-income minority students above the current public school level. The problem is that we lack a system for reliably replicating the good schools and crowding out the rest. And what kind of problem is that? Even Wikipedia knows the answer:

Engineering is the discipline, art and profession of acquiring and applying scientific, mathematical, economic, social, and practical knowledge to design and build… systems… and processes that safely realize solutions to the needs of society.

Engineering is just a broad set of tools for finding practical solutions to complex problems. One of the most useful of those tools is an aversion to reinventing the wheel, so engineers always ask how the kind of problem they’re addressing has been approached previously, in other places, even in other fields. When possible, they adapt proven solutions to the problem at hand.

So let’s all be engineers for a day on January 28th and hear what education experts from Sweden and Chile have to say about how their nations have been encouraging the replication of good schools. You can register for this unique lunchtime event here.

Topics: 
Education and Child Policy