Tag: college board

Now You Can Draw Meaningful Time Trends from the SAT. Here’s How…

Over the years, countless reporters and even policy analysts have attempted to draw conclusions from changes in state SAT scores over time. That’s a mistake. Fluctuations in the SAT participation rate (the percentage of students actually taking the test), and in other state and student factors, are known to affect the scores.

But what if we could control for those confounding factors? As it happens, a pair of very sharp education statisticians (Mark Dynarski and Philip Gleason) revealed a way of doing just this—and of validating their results—back in 1993. In a new technical paper I’ve released this week, I extend and improve on their methods and apply them to a much larger range of years. The result is a set of adjusted SAT scores for every state reaching back to 1972. Vetted against scores from NAEP tests that are representative of the entire student populations of each state (but that only reach back to the 1990s), these adjusted SAT scores offer reasonable estimates of actual changes in states’ average level of SAT performance.

The paper linked above reveals only the methods by which these adjusted SAT scores can be computed, but next week Cato will publish a new policy paper and Web page presenting 100 charts—two for each state—illustraing the results. How has your state’s academic performance changed over the past two generations? Stay tuned to find out…

Update: Here’s the new paper and charts!

The SAT Commits Suicide

The College Board announced this week that it is dropping the more arcane words and more advanced mathematics from its SAT test, among other changes. This, however noble its intentions, seems counterproductive and institutionally suicidal.

The purpose of the SAT is to help predict success in college. It does this in the same way as every other test: by distinguishing between those who know the tested content and those who do not. Not surprisingly, most modern tests are designed using something called “Item Discrimination Analysis.” That unfortunately-named technique has nothing to do with racism or classism. It is simply a mathematical formula. What it does is measure, for every question, the difference between the percentage of high-performers who got the question right and the percentage of low-performers who got it right. In general, the higher this “Discrimination Index” (DI) rises, the more useful the question is and therefore the more likely it is to be retained.

The problem with the College Board’s announced revisions is that they seem likely to eliminate questions with high DI values in favor of others with lower DI values. You might guess that reducing the SAT’s ability to distinguish between high and low performers would inhibit its ability to predict college success. But you don’t have to guess, because there’s already at least one recent study that looked at this question. What the authors found is that the DI value of SAT mathematics questions is usually the strongest contributor to the test’s ability to predict college success—by a wide margin.

There’s a good chance that the College Board is aware of this study since two of its three authors work for the College Board and the Board hosts a presentation about the study on its own website.

The Board’s changes are intended to make the SAT more fair. In practice, they seem likely to make it less useful. And as its usefulness diminishes, so will the number of colleges using it. If this proves to be the case—and we’ll know for sure in just a few years—the College Board will have succeeded in doing something that its critics have been unable to accomplish despite decades of effort: killing the SAT.

College Board’s SAT Drop Spin Doesn’t Hold Up

Nationwide verbal SAT scores fell to their lowest level in years on the most recent administration of the test, and the College Board, which administers the SAT, has an explanation:

Average SAT scores fell slightly for 2011 high-school graduates, as the number of test takers and the proportion of minority students grew, according to a report released on Wednesday by the College Board, which owns the test.

The idea—which has been offered as an explanation of earlier declines—is that the overall average score can fall even if the performance of every participating group was stable or improving—if the groups that tend to score lower comprise a larger share of the total test-taking population than they did in the past. And, indeed, minority students (who often score below white students) now comprise a larger share of the test taking population than ever before.

So: case closed? Nope. If you actually look at the score breakdown for the major race/ethnicity groups (see chart) you’ll notice that only white students’ scores held constant from last year. The scores of all the minority groups declined. And, since 1996, white students’ scores have been flat, those of Asian students have risen appreciably, and those of Hispanic and African American students have declined.

Since there has not been any government program targeted exclusively at improving the achievement of Asian students, these data don’t exactly bolster confidence in the effectiveness of either state or federal education policy. If we want to see improved educational productivity, we might just want to look at more free enterprise education systems that offer schools the freedoms and incentives that actually make it happen.

The Student Aid Did It!

The College Board is out with its annual reports on college prices and student aid, and the story is pretty familiar. According to The New York Times, the reports reveal that over the last year tuition and fees rose 8 percent at public, four-year schools, and 4.5 percent at private non-profits. Meanwhile, student aid rose at a very fast clip. Indeed, over the last five years, despite lightning-quick growth in sticker prices, after-aid college costs actually dropped.

Now, don’t expect to hear this from the College Board or even mentioned in the Times, but doesn’t it seem at least plausible that giving more and more aid to students enables schools to raise prices? You know, that colleges might jack up tuition and fees knowing that government, largely, will ensure that students can cover them? It’s not only plausible, it’s almost certainly the case. But like I said, forget about ever reading that in The New York Times. Instead, we get this standard lament:

“The College Board figures are depressing and utterly predictable,” said Terry Hartle, senior vice president of the American Council on Education. “When states cut funding for higher education, tuitions go up to make up for the difference.”

Dealing with this one gets incredibly tiresome, and it should infuriate taxpayers who fund both massive student aid and subsidies to public colleges.

For one thing, of course, cuts in  state subsidies don’t explain constantly increasing private school costs. Moreover, while no doubt public schools sometimes raise tuition to make up for state funding dips, they also raise it when state funding is going up. Indeed, as this chart from the State Higher Education Executive Officers illustrates, public schools raise prices no matter what is going on with state and local subsidies:

How can schools get away with this? Because students are able to cover the incessantly rising prices. And how can students do that? By using more and more money that comes from someone else!

The data scream this reality so loudly even passed-out undergrads could hear it. So why does it get so little attention? In part, no doubt, because many in the media refuse to even consider that there could be a causal connection between ballooning aid and skyrocketing prices. Even worse, the people controlling the aid see votes, votes, votes from playing education Warbucks. And if, say, the President of the United States can buy votes with student aid, why would he ever admit that his “generosity” mainly just lets higher education bleed taxpayers dry? The unfortunate answer is, he wouldn’t.

Nothing Good about The Higher Ed Pricing Game

On Tuesday I noted that the College Board had released its annual reports on college prices and student aid. At the time I wrote the post I hadn’t yet been able to download the reports, but was planning to provide a rundown of their major findings once I’d read them. I’ve now done the latter, but it turns out that Ben Miller over at the Quick and the ED has already posted a pretty good summary of the most important findings. Go there if you want the highlights. Don’t go there, though, if you want to know what the highlights mean, at least for anyone other than students. For that, you’ll have to read on here….

The big news is that net college prices – what students pay after aid– have actually decreased over the last 15 years. While sticker prices were rising much faster than incomes and inflation, what students were actually paying dropped. The implication of this is so obvious that Mr. Magoo couldn’t mistake it: Student aid, much of which comes through taxpayers, enables schools to charge ever-higher prices with near impunity.

Back to the Quick and the ED. To some degree, Miller sees declining net price as a triumph for federal aid, making college more affordable even as prices explode:

This story should be encouraging for legislators that fought hard to win Pell Grant increases over the last few years. The steepest decreases in net price occur beginning in the 2007-2008 academic year, the same time Congress began passing legislation that boosted the maximum Pell Grant award several times. This at least suggests that the money spent on the program did play some role in lessening the financial burden for students and was not completely eaten up by sticker price increases.

On the flip side, Miller at least acknowledges that:

The net price figure also lessens the pressure on schools to actually take proactive steps to lower their costs. If the price you list isn’t actually what you charge, then why should anyone care what the listed price is and how high it gets? Net price thus serves as a kind of smokescreen that gets colleges at least partially off fo[r] charging an arm and a leg.

So what’s wrong with this analysis? 

Most important is that Miller softpedals the aid effect, suggesting that the main negative consequence of  ever-increasing assistance is that it bleeds off a bit of the pressure for schools to lower costs. But it likely has a much more destructive effect than that, not just curbing efficiency pressures, but enabling schools to constantly charge and spend more.  It’s a likelihood that student-aid defenders try to dispel by citing studies that cover very short periods of time, or that simply pronounce that we don’t know that it happens. That it probably happens, however, has been borne out empirically, and it’s readily ackowledged by prominent higher educators including former Harvard president Derek Bok, former Stanford vice president William F. Massy, and former University of Iowa president Howard Bowen. Indeed, the latter’s “law” couldn’t be more blunt: “Universities will raise all the money they can and spend all the money they raise.”

Miller’s other major failing is that he completely ignores that all this aid has to come from somwhere, and that “somewhere” is largely taxpayers. (OK, first it’s China.) Just to give you a sense of the impact on taxpayers, College Board data show that between the 1998-99 and 2008-09 academic years, total federal aid – including grant money recipients don’t have to pay back, and loans they (sometimes) do – rose from $61.1 billion to $116.8 billion. Add state aid to that, and the total goes from $66.6 billion to $126.2 billion.

And what are some of the major downsides of these forced third-party payments? Miller mentions a few pricing difficulties for students, but makes no mention of the potentially huge negative consequences for the nation: Encouraging lots of people to attend college who simply aren’t prepared for it; cranking out many more degrees than the job market demands; and potentially slowing economic growth by taking funds from productive uses and giving it to efficiency-averse colleges and students. 

The big finding in the latest College Board data, which the Quick and the ED nails, is that net college prices have been going down. The important story, however, is that this is bad news for the country. Unfortunately, the Quick and the Ed misses that almost completely.

College Prices Aren’t So Bad When Other People Are Paying

Today the College Board – maker of such fine products as the SAT and Advanced Placement exams – released its annual reports on college prices and student aid. College prices, it seems, have gone up significantly over the last year. However, if the following statement from the reports’ author, economist Sandy Baum, is accurate – I haven’t been able to see the reports myself yet – student aid largely offset the price increases. And do you know what that might mean? Colleges were able to charge students more without greatly affecting access by pawning much of the new charges off on donors and taxpayers:

Sandy Baum, the College Board senior policy analyst who wrote both reports, said it was important to focus on the net price students actually paid, after subtracting grants and tax benefits, rather than the published tuition, or sticker price. And in that regard, Ms. Baum said, the situation looks far less dire. “Over all, it could have been worse,” she said.

So could it actually be, as I and others have argued repeatedly, that student aid helps fuel tuition increases by having third parties cover so much of the new costs? Here’s yet more evidence saying that yes, it could.