Tag: discrimination analysis

The SAT Commits Suicide

The College Board announced this week that it is dropping the more arcane words and more advanced mathematics from its SAT test, among other changes. This, however noble its intentions, seems counterproductive and institutionally suicidal.

The purpose of the SAT is to help predict success in college. It does this in the same way as every other test: by distinguishing between those who know the tested content and those who do not. Not surprisingly, most modern tests are designed using something called “Item Discrimination Analysis.” That unfortunately-named technique has nothing to do with racism or classism. It is simply a mathematical formula. What it does is measure, for every question, the difference between the percentage of high-performers who got the question right and the percentage of low-performers who got it right. In general, the higher this “Discrimination Index” (DI) rises, the more useful the question is and therefore the more likely it is to be retained.

The problem with the College Board’s announced revisions is that they seem likely to eliminate questions with high DI values in favor of others with lower DI values. You might guess that reducing the SAT’s ability to distinguish between high and low performers would inhibit its ability to predict college success. But you don’t have to guess, because there’s already at least one recent study that looked at this question. What the authors found is that the DI value of SAT mathematics questions is usually the strongest contributor to the test’s ability to predict college success—by a wide margin.

There’s a good chance that the College Board is aware of this study since two of its three authors work for the College Board and the Board hosts a presentation about the study on its own website.

The Board’s changes are intended to make the SAT more fair. In practice, they seem likely to make it less useful. And as its usefulness diminishes, so will the number of colleges using it. If this proves to be the case—and we’ll know for sure in just a few years—the College Board will have succeeded in doing something that its critics have been unable to accomplish despite decades of effort: killing the SAT.