Tag: bias

Author of the Private School Spending Study Responds

Bruce Baker, author of the study of private school spending about which I blogged yesterday, has responded to my critique. Dr. Baker thinks I should “learn to read.”

He takes special exception to my statement that he “makes no serious attempt to determine the extent of the bias [in his chosen sample of private schools], or to control for it.” Baker then points to the following one paragraph discussion in his 51 page paper that deals with sample bias, which I reproduce here in full [the corresponding table appears on a later page]:

The representativeness of the sample analyzed here can be roughly considered by comparing the pupil-teacher ratios to known national averages. For CAS and independent schools, the pupil-teacher ratio is similar between sample and national (see Figure 21, later in this report). Hebrew/Jewish day schools for which financial data were available had somewhat smaller ratios (suggesting smaller class sizes) than all Hebrew/Jewish day schools, indicating that the mean estimated expenditures for this group might be high. The differential, in the same direction, was even larger for the small group of Catholic schools for which financial data were available. For Montessori schools, however, ratios in the schools for which financial data were available were higher than for the group as a whole, suggesting that estimated mean expenditures might be low.

Even with my admittedly imperfect reading ability, I was able to navigate this paragraph. I did not consider it a serious attempt at dealing with the sample’s selection bias. I still don’t. In fact, it entirely misses the main source of bias. That bias does not stem chiefly from class size differences, it stems from the fact that religious schools need not file spending data with the IRS, and that the relatively few that do file IRS Form 990 (0.5% of Catholic schools!) have a very good reason for doing so: they’re trying harder to raise money from donors.  This is not just my own analysis, but also the analysis of a knowledgeable source within Guidestar (the organization from which Baker obtained the data), whose name and contact information I will share with Dr. Baker off-line if he would like to follow-up.

Obviously, schools that are trying harder to raise non-tuition revenue are likely to… raise more non-tuition revenue. That is the 800 pound flaming pink chihuahua in the middle of this dataset. According to the NCES, 80 percent of private school students are enrolled in religious schools (see p. 7), and this sample is extremely likely to suffer upward bias on spending by that overwhelming majority of private schools. They may spend the extra money on facilities, salaries, equipment, field trips, materials, or any number of other things apart from, or in addition to, smaller classes.

Baker’s study does not address this source of bias, and so can tell us nothing reliable about religious schools, or private schools in general, either nationally or in the regions it identifies. The only thing that the study tells us with any degree of confidence is that elite independent private schools, which make up a small share of the private education marketplace, are expensive. An uncontroversial finding.

It is surprising to me that this seemingly obvious point was also missed by several other scholars whose names appear in the frontmatter of the paper. This is yet another reminder to journalists: when you get a new and interesting paper, send it to a few other experts for comment (embargoed if you like) before writing it up. Doing so will usually lead to a much more interesting, and accurate, story.

PASS ID and National ID - Rejoinder to Schwartz

Ari Schwartz responded in characteristic even tones to my critique of his testimony in favor of the PASS ID Act, which would revive the moribund REAL ID law. It’s worth a rejoinder, and I’ll offer him the same again here if he wishes.

Ari clouds matters slightly by suggesting that my “strong biases” obscure certain facts. I readily admit having a strong bias in favor of liberty – it’s why I do what I do. Ari admits several biases, including one in favor of consensus-building, which was what I accused him of prioritizing over principle. Let’s put aside the question of bias.

It’s good to see Ari state that CDT does not support a national ID system. It would be better to see him state that CDT opposes having a national ID system. (I imagine this is just a matter of word choice, but it would be good to have clarity.)

Next, Ari says his testimony “makes it clear that we believe that PASS ID prevents the creation of a National ID system.” I don’t believe this is clear from his testimony. More importantly, this is not a sound assessment of what a national ID is or what PASS ID does.

We need some defined terms, so let’s tease out what he means by “national ID.” (He has told me that there is some distinction between a “national ID,” a “national ID system,” and perhaps a “national ID card,” but the distinction is lost on me. I believe a national ID card is part of a national ID system, both of which are commonly referred to in shorthand as a “national ID.”)

Twice in his testimony, he correctly calls REAL ID a national ID system. The factors that make it so appear to be “the very real possibility that individuals would not be able to function in American society without a REAL ID card” and “giving unfettered discretion to DHS to expand the ‘official purposes’ for which REAL ID cards could be required.”

In my recent post on the subject, I defined a national ID as being a card: 1) nationally uniform in its key elements; 2) the possession of which is either practically or legally required; and 3) that is used for identification.

I think 1) and 3) are both given. Ari’s take on 2) - inability to function without it – and my formulation – practically required – are equivalent, so Ari and I agree on that much.

But is DHS discretion to expand “official purposes” an essential element of a national ID card? I don’t think so.

Let’s say Congress passes a law requiring employers to check a certain card before they hire new workers. What if Congress requires credit issuers to check the card? States require presentation of the card at the voting booth? What if Congress requires pharmacists to check it before selling people cold medicine?

Is this card system saved from being a “national ID system” because someone other than DHS came up with these ideas? Of course not. DHS discretion to expand usage is not what makes an ID system a “national ID system.”

The better definition is what we agree on: A national ID is national, identifying, and practically or legally required, meaning the lack of it disables people from functioning in society.

Do REAL ID and PASS ID differ in ways that make the one a national ID and the other not a national ID? No, and Ari doesn’t say so. He merely says PASS ID would slow national ID mission creep by some margin because it denies DHS some discretion. (PASS ID “[r]emoves from DHS’s authority the ability to unilaterally determine new official purposes for which a PASS ID-compliant card can be required … .”)

This is not central to “national ID-ness,” and PASS ID doesn’t actually deny DHS that authority – it simply removes the specific grant of authority in REAL ID. Removing a grant of authority in one law does not deny an agency authority it has elsewhere. (It’s like the difference between “not supporting” and “opposing” something.) DHS and other agencies almost certainly have power under other law to require the IDs they choose for functions that are plausibly related to security or fraud prevention.

I was wrong to assume that it was lack of principle driving CDT and Ari to endorse the PASS ID Act, which revives our moribund national ID law. Other explanations are no more palatable, though, and no other group that I am aware of missed the true import of PASS ID.

Here’s a memorable Bruce Schneier quote to emphasize the importance of opposing a national ID, which so many civil liberties groups are doing:

History will record what we, here in the early decades of the information age, did to foster freedom, liberty and democracy. Did we build information technologies that protected people’s freedoms even during times when society tried to subvert them? Or did we build technologies that could easily be modified to watch and control? It’s bad civic hygiene to build an infrastructure that can be used to facilitate a police state.

No civil liberties group supports PASS ID. CDT can’t claim that mantle while it does.