For example, an experimental evaluation of a private school voucher program in Washington, D.C. found that winning the lottery to attend a private school had mixed effects on test scores, but over a 50 percent increase on students’ tolerance of others. Another rigorous evaluation of a voucher program in Milwaukee found no effects on reading test scores after four years. On the other side of the equation, my coauthor and I followed the same sample of students until they were around 22 to 25 years old and found that the Milwaukee program reduced the likelihood that they committed crimes as adults by over 50 percent. Other studies found significant divergences between test scores and outcomes such as charitable giving, political participation, effort, and happiness in school.
And divergences exist outside of the school choice literature as well. At least five studies that I know of using rigorous value‐added methodology find disconnects between teachers’ effects on student test scores and their effects on student character skills such as behavior and effort. For example, Northwestern University professor Kirabo Jackson finds that teachers’ effects on student behavior are much stronger predictors of high school graduation than their effects on student test scores. Indeed, Jackson finds that teachers’ effects on student behavior are over 8 times as influential for high school graduation as effects on student test scores.
Of course, we cannot ignore the fact that some studies do find a link between test scores and long‐term outcomes. For instance, prominent education scholars – Chetty, Friedman, and Rockoff — found that teachers that improve student test scores also tend to have positive effects on earnings later on in life. But that does not at all mean that their effects on test scores caused the effects on earnings. It is more likely that — in their sample — teachers that were good at shaping test scores were also good at shaping the non‐cognitive skills necessary for success in the long‐run, on average.
In other words, teachers that are good at improving standardised test scores can also be good at motivating students to work hard and to treat others with respect. Hard work and respect may be the skills that influence long‐run outcomes such as earnings. But it is nearly impossible to accurately measure soft skills, which could be why we have found so many divergences in the literature.
Either way, what appears clear is that focusing too much on test scores could compromise the character development necessary for true lifelong success.