Tag: guidestar

Author of the Private School Spending Study Responds

Bruce Baker, author of the study of private school spending about which I blogged yesterday, has responded to my critique. Dr. Baker thinks I should “learn to read.”

He takes special exception to my statement that he “makes no serious attempt to determine the extent of the bias [in his chosen sample of private schools], or to control for it.” Baker then points to the following one paragraph discussion in his 51 page paper that deals with sample bias, which I reproduce here in full [the corresponding table appears on a later page]:

The representativeness of the sample analyzed here can be roughly considered by comparing the pupil-teacher ratios to known national averages. For CAS and independent schools, the pupil-teacher ratio is similar between sample and national (see Figure 21, later in this report). Hebrew/Jewish day schools for which financial data were available had somewhat smaller ratios (suggesting smaller class sizes) than all Hebrew/Jewish day schools, indicating that the mean estimated expenditures for this group might be high. The differential, in the same direction, was even larger for the small group of Catholic schools for which financial data were available. For Montessori schools, however, ratios in the schools for which financial data were available were higher than for the group as a whole, suggesting that estimated mean expenditures might be low.

Even with my admittedly imperfect reading ability, I was able to navigate this paragraph. I did not consider it a serious attempt at dealing with the sample’s selection bias. I still don’t. In fact, it entirely misses the main source of bias. That bias does not stem chiefly from class size differences, it stems from the fact that religious schools need not file spending data with the IRS, and that the relatively few that do file IRS Form 990 (0.5% of Catholic schools!) have a very good reason for doing so: they’re trying harder to raise money from donors.  This is not just my own analysis, but also the analysis of a knowledgeable source within Guidestar (the organization from which Baker obtained the data), whose name and contact information I will share with Dr. Baker off-line if he would like to follow-up.

Obviously, schools that are trying harder to raise non-tuition revenue are likely to… raise more non-tuition revenue. That is the 800 pound flaming pink chihuahua in the middle of this dataset. According to the NCES, 80 percent of private school students are enrolled in religious schools (see p. 7), and this sample is extremely likely to suffer upward bias on spending by that overwhelming majority of private schools. They may spend the extra money on facilities, salaries, equipment, field trips, materials, or any number of other things apart from, or in addition to, smaller classes.

Baker’s study does not address this source of bias, and so can tell us nothing reliable about religious schools, or private schools in general, either nationally or in the regions it identifies. The only thing that the study tells us with any degree of confidence is that elite independent private schools, which make up a small share of the private education marketplace, are expensive. An uncontroversial finding.

It is surprising to me that this seemingly obvious point was also missed by several other scholars whose names appear in the frontmatter of the paper. This is yet another reminder to journalists: when you get a new and interesting paper, send it to a few other experts for comment (embargoed if you like) before writing it up. Doing so will usually lead to a much more interesting, and accurate, story.

Union-Funded Study Says Private Schools Expensive!

I know, it’s a bit of a dog-bites-man headline, but bear with me. A new study by a Rutgers University ed. professor purports to tell us about “Private Schooling in the U.S.: Expenditures, Supply, and Policy Implications.” The trouble is, the study presents no data that are representative of private schooling in the U.S.

Author and ed school professor Bruce Baker analyzed per pupil expenditures of private schools that had registered with Guidestar.org. Based on its mission statement, Guidestar is a service brings together charities seeking donations with would-be donors, in an effort to encourage philanthropy. Only a fraction of the nation’s private schools participate, and they are self-selected into that group. It is reasonable to think that the schools that self-select into Guidestar are the ones most avidly seeking donations. According to a PowerPoint presentation on Guidestar’s site, its top five types of users are:

  • Non-Profit Development Directors
  • Non-Profit Fundraising Directors
  • Grant Writers
  • Foundation Grants Administrators and Donor Services Managers
  • Corporate Foundation Giving Program Managers

Quite possibly, the private schools most actively seeking non-tuition revenue are the ones… receiving the most non-tuition revenue. So not only is the Guidestar population of private schools not randomly selected, and non-representative of private schools nationally, there is reason to believe it is biased in the direction that its author and funders favor.

This would be bad enough, but it gets worse. The author makes no serious attempt to determine the extent of the bias, or to control for it. In fact, he consciously makes it worse: he choses to eliminate from consideration any private schools reporting revenues or expenditures under $500,000, thereby excluding smaller, less expensive schools.

I have literally NEVER seen a serious academic study that starts from a sample that is known to be biased in the direction favored by its funders and then consciously makes matters worse by actively skewing it even further!

An example of the kind of analysis that is supposed to accompany the presentation of a non-random sample to ascertain extent and direction of bias appears in my own 2006 study of Arizona private schools, available here. I dedicate five pages (beginning on page 14) to an assessment of whether and to what extent my survey respondents differed from the universe of all Arizona private schools. Significant effort was expended on that section of the study, because it is both necessary and expected. I was disappointed, though not surprised, by the absence of such a section in the Baker study.

Not only can the Baker study not tell you how much U.S. private schools really spend, it seems to have a little difficulty getting the public school spending figures right, too. For instance, there is a line on page 42 implying that DC public schools were spending $14,000 in 2007.  Federally-reported data show that DC was already spending over $18,000 per pupil in 2005-06. And I’ve shown that it spent $28,000/pupil in 2008-09.

Finally, did I mention that Baker’s study was funded by the NEA-bankrolled “Great Lakes Center for Education Research and Practice”? As Ed Sector pointed out a couple of years ago: “The Great Lakes Center and the NEA’s Michigan affiliate are also linked on a personal level: [the Center’s director] Teri Battaglieri is married to Michigan Education Association Executive Director Lou Battaglieri.”

***

Update:  Note that the reason Guidestar only has financial information for a small fraction of the nation’s private schools is that the vast majority of U.S. private schools are religious, and religious schools are not required to file IRS Form 990 (from which Guidestar gets its financial data). The religious private schools that do file Form 990 are thus a small self-selected group that is presumably seeking to maximize its revenue from charitable donations, and hence very likely biased toward higher spending schools.