Topic: Education and Child Policy

We’re #1!

Jay Greene—a man known for producing some of the most interesting (and voluminous) education research in the country—today published the most important finding of his career: When it comes to readership, Cato [at] Liberty.org crushes all other education blogs!

OK, as Greene points out, Cato@Liberty is not strictly an education blog. (I have little doubt that readers come for the brilliant education coverage first and consider the rest of the content mainly an added — but outstanding! — bonus.) Even if that’s not the case, we thank all of you who take the time to read our education posts and want to tell lesser education bloggers that we think you’re very nice. And who knows, maybe some day In-Bev will buy us and someone else can be the king.

Pre-K Pushers Peddling Patent Prevarications

We left off in our last episode with a teaser on why the blessed Perry Preschool Project, Carolina Abecedarian Project, and Chicago Child-Parent Centers Program don’t prove what many people like to pretend they prove; the long-term effectiveness, let alone cost-effectiveness, of current large-scale preschool programs.

Let’s begin with the fact that these programs were all small-scale, intensive, and targeted at the most disadvantaged children.

Now let’s look at some highlights of the other problems with using these as evidence in support of government preschool programs:

The Perry Preschool Project was an early-education intervention “experiment” initiated by researchers at the High/Scope Educational Research Foundation in 1962 and concluded in 1965. Project researchers analyzed the effect of home visits and preschool on outcomes for an unusual sample of 58 low-income children with IQs between 70 and 85 compared to a “control” group of 65 other children who did not attend the preschool program or receive home visits.

Researchers concluded that the positive effects of the program on outcomes such as future earnings and crime rates far outweighed the costs, giving taxpayers a return of $7.16 for every dollar invested. However, there are a number of problems with the Perry Preschool Program and the associated analyses that render them unreliable and unsuitable for estimating the effects of the large-scale programs currently under consideration in state legislatures:

One analysis noted that “the Perry Project poses a number of methodological difficulties” that call into question any conclusions about the effects of the program, positive or negative. Assignment to the preschool treatment and “control” groups was not completely random — an absolute requirement for valid experiments. And the children in the preschool program had to have parents home during the day — a requirement “resulting in a significant difference between control and intervention groups on the variable of maternal employment” that also calls into question any results.

Other criticisms included:

  • Twisting the Data: Program researchers expanded the standard definition of “statistical significance” in order to find positive effects. Most effects disappeared when the scientific standard was used.
  • More than Preschool: The program included home visitations in addition to preschool, which made it difficult if not impossible to determine whether preschool alone had significant positive effects.

The Abecedarian Project was an intensive early-intervention program begun in 1972 that placed participating infants, who were on average at 4.4 months old, “in an eight-hour-a-day, five-day-per-week, year-round educational day care center” where they “received free medical care, dietary supplements, and social service support for their families.” Half of the children in this intensive infant intervention program received three more years of educational assistance, as did half of the children in the control group that did not participate in the intervention program as infants and toddlers.

Project researchers found that the infants who received the intensive early intervention scored higher than the control children on cognitive and academic tests at age 12 and 21. There are, however, a number of problems with the Abecedarian Project and the associated analyses that render them unreliable and unsuitable for estimating the effects of the large-scale programs currently under consideration in state legislatures:

  • Much More than Preschool: The most obvious and serious problem with this “preschool” program is that the intervention was nothing like the preschool programs currently being considered or in effect. The Abecedarian Project was an intensive, long-term intervention beginning in infancy, and it can therefore shed little if any light on the effects of preschool on 3 or 4 year-olds.
  • Methodological Problems: Studies reporting effects from the Abecedarian Project generally focus on the differences between the treatment and control groups at later ages. Herman H. Spitz, a well-respected academic psychologist specializing in measuring intelligence among those with developmental disability, notes that the advantage found later emerged when the children were just 6 months old. Spitzer rightly concludes, “We need to understand why an additional 4.5 years of intensive intervention had so little effect that, at six years of age (and older), the difference between the intervention and control groups was not appreciably different than it had been at six months of age.”

The Chicago Child Parent Center Program was an early-education and family intervention begun in 1985 involving 989 low-income children in Chicago. Researchers concluded that, compared to the 550 children who did not receive the intervention, children in the program had a “higher rate of high school completion; more years of completed education; and lower rates of juvenile arrest, violent arrests, and school dropout.” This study has been used in recent years by a RAND analysis that claims a universal preschool program would return $2.62 for every dollar invested.

Again, however, there are a number of problems with the Chicago Child Parent Center Program and the associated analyses that render them unreliable and unsuitable for estimating the effects of the large-scale programs currently under consideration in state legislatures:

  • More than Preschool—Parenting: As the name of the program implies, the Chicago Child Parent Center Program involved extensive interventions with parents that involved “a multifaceted parent program that includes participating in activities in the parent resource room with other parents (e.g., educational workshops, reading groups, and craft projects), volunteering in the classroom, attending school events and field trips, and completing high school; outreach activities including resource mobilization, home visitation, and enrollment of children.”
  • More than Preschool—Tutoring: The intervention continued through 3rd grade for some students, and involved tutoring, speech therapy, and medical services that are not a part of current preschool proposals and significantly raise the costs and difficulties of expanding to a state-wide program.
  • Wild Extrapolations: The RAND study does not consider these important concerns regarding the Chicago Child Parent Center Program. Instead they uncritically apply the findings from this intensive family intervention program to a state-wide, universal preschool-only program. The researchers also arbitrarily assign middle and upper-income children benefits from preschool that no study of the Chicago Child Parent Center Program suggests they receive.

The fourth and final post is coming soon, with a response to some other objections …

Barack Obama Walks the Walk

After telling a gathering of the American Federation of Teachers that he opposes school voucher programs over the weekend, Senator Obama added that: “We need to focus on fixing and improving our public schools; not throwing our hands up and walking away from them.”

Senator Obama sends his own two daughters to the private “Lab School” founded by John Dewey in 1896, which charged $20,000 in tuition at the middle school level last year. Though he says “we” should not be “throwing up our hands and walking away” from public schools, he has done precisely that.

That is his right, and, as a wealthy man, it is his prerogative under the current system of American education, which allows only the wealthy to easily choose between private and government schools. But instead of offering to extend that same choice to all families, Senator Obama wants the poor to wait for the public school system to be “fixed.”

I could editorialize about this, but I really don’t see the need. Readers of this blog are perfectly capable of drawing the obvious conclusions.

Teaching to the Test

A lot of people dislike No Child Left Behind–style test-driven reforms because they fear that schools will “teach to the tests.” That is, the schools will focus on content that will likely appear on tests, as well as teach strategies to game specific assessment tools, rather than effectively teach the broad content and understanding that tests, ideally, should merely sample in order to gauge student mastery. If this were the case, it would both severely constrict how much of educational value kids actually learn and call into serious question whether improved test scores really signify improved learning.

Whether or not this “teaching to the test” regularly happens is a highly debatable — and debated — matter. Reading today’s Washington Post article about considerable one-year improvements in D.C. test scores, however, certainly gives you pause to think that when test results are almost all that schools are judged on, mastery of the tests — not the subjects — could truly end up being all that really matters.

Pre-K Pushers Possess Paltry Proof of Preschool Payoff

My first pre-k post showed that Oklahoma’s 4th grade reading NAEP scores have dropped and stagnated compared to the national average, and that changes in poverty levels and per-capita income can’t explain why we don’t see improvement from the state’s model investment in preschool.

Regardless of the evidence that there is little or no long-term effect from preschool, the critics will always point to the remaining shadowy corners of uncertainty. With so many possible confounding variables, it is impossible to control for them all. There might be some hidden, overlooked factor that canceled out the real, substantial long-term effects.

That’s correct. Highly unlikely in the case of the spectacular absence of a return on Oklahoma’s preschool investment and no obvious alternative explanation. But possible nonetheless.

And that is why non-experimental analysis can only provide suggestive evidence, with a heavy dose of uncertainty. Among the available research methods, the only way to be fairly certain an educational treatment has had an effect on students is to conduct a controlled experiment akin to those used in medicine or drug testing. Researchers randomly assign each person to either get the treatment or to not get the treatment.

Unfortunately, the preschool pushers have no experimental evidence that the pre-k programs they promote have a significant, long-term, positive effect.

That’s why they rely so heavily on the few pieces of experimental evidence from programs that look nothing like those in Oklahoma, Georgia, or any other state that has adopted or is considering a pre-k program.

Preschool activists kneel before a holy trinity of early-intervention programs that supposedly prove preschool is our educational, nay … our societal savior: the Perry Preschool Project, Carolina Abecedarian Project, and Chicago Child-Parent Centers Program (Sara inexplicably forgot to mention the Abecedarian Project).

Unfortunately, they don’t come even close to proving what preschool activists pretend they do.

To find out why, stay tuned … .  (and yes, I’m sticking with the excessive title alliteration.)

Education Journalists: Free Markets Demand an Apology

What do you need to have a free market in education? Price change? Low barriers to entry? Product differentiation? Other good stuff? Nah! All you need is something slightly less restrictive than traditional, absolute command-and-control public schooling, and you’ve got and yourself a free market, Bub!

OK, obviously that’s not the case, but you wouldn’t know it from education stories you read in some of the nation’s biggest newspapers. A couple of weeks ago the Washington Post ran an article that called the hiring of private management firms to run some public schools in Philadelphia a test of whether the “free market could educate children more efficiently than the government.” Andrew Coulson took the Post to task for that, pointing out that “Philadelphia did not create a ‘free market’ in education. What it did was to subcontract aspects of its monopoly to providers of its own choosing.”

Today, the Houston Chronicle offers an arguably even more egregious abuse of the term “free market,” declaring that allowing some kids from outside of the Houston Independent School District to go to Houston schools free of charge—but carrying state dollars—“would employ a free market approach to increase revenue while addressing the needs of students just beyond the HISD boundary.” No new providers—not even new managers, like in Philly—no pricing, no product differentiation, just a few kids able to attend government schools in Houston rather than in their home districts.

It’s hard to tell whether this regular sullying of the free market’s good name is done to make free markets look bad, or out of ignorance. If it’s the former, then Houston (I can’t help myself!) we have a problem! If it’s the latter, thankfully we have a solution: Education journalists, give this kid a call!

Pre-K Pusher Pans Preschool Pessimist

Sara Mead of the New America Foundation, one of the growing number of pre-K pushers, takes issue with my pointing out that Oklahoma’s NAEP scores suggest no return on their massive and celebrated investment in preschool over the past 18 years.

This is just one small item in a box full of evidence that suggests preschool has at best a negligible impact on long-term student outcomes. There are a lot of problems with the edifice of misinformation and misunderstanding that the preschool activists have built. So, this will be an epic four-part series of posts. It will test my resolve and yours, but we must sacrifice for the greater good. Can you handle this much pre-k?

Sara correctly points out that the fact that Oklahoma’s performance has fallen and then stagnated compared to the national average is not definitive proof that pre-k failed to have a massive positive impact on student performance; many things could have happened to cancel out improvements from preschool … like a massive influx of Hispanic immigrants, or any number of changes in the educational system. (Considering the sloth-like speed of the government school system in executing any substantive change, I propose that this last concern be dismissed outright.)

So let’s take a look at some of the big factors that could have wiped out the huge academic boost preschool activists claim pre-k provides; income levels, poverty rates, and Hispanic population.

The percentage of students of Hispanic origin in Oklahoma is still very low – at eight percent, it’s less than half the national average of 19 percent. So let’s turn to the most important factors correlated with student performance, income and poverty:

We can see here that in Sara’s favorite pre-k impact subject, reading, Oklahoma has not improved at all compared to the national average despite a massive and acclaimed investment in government pre-k (and the nation as a whole has actually declined in its performance on international tests relative to other wealthy nations).

Oklahoma’s poverty rate has bounced up and down around an average of 16 percent higher than the national average, with no trend at all. Per-capita income has stayed at least ten percent lower than the national average but has trended ever so slightly higher.

So I challenge you, Sara, and any other preschool activist out there, to find the nefarious factor that has destroyed all the gains from pre-k. By all means, take this data and run it through statistical software with whatever controls you’d like related to documented demographic and education changes (as long as you include the national averages as a control). I’d do it myself, but I’m sufficiently convinced already that the null hypothesis won’t be rejected.

I offer a gentleman’s bet that you’ll find no significant positive correlation between the number of children attending pre-k and NAEP scores in 4th or 8th grades.

More on the poverty of preschool claims soon …