About that National Journal gun chart…

Isenstein’s chart has since been posted on countless blogs, Twitter feeds, and Facebook pages, with the subtext (and often the explicit text) that if troglodyte gun‐​rights supporters could appreciate simple statistics, they’d stop impeding common‐​sense gun controls that would deter terrible crimes like the one in Roseburg. President Obama also made this point explicit last week in a statement about the Roseburg shooting:

The president’s comment has since received some critical scrutiny from The Washington Post’s “Fact‐​Checker” Glenn Kessler. And UCLA law professor and Post contributor Eugene Volokh criticized the National Journal table in a recent post on the lack of a statistical link between gun control and overall homicide rates. Yet the chart continues to bounce around the Internet and the media. Its currency reveals how readily people seize on statistics they don’t really understand but think—in this case, wrongly—support their opinions, claiming the intellectual high ground while dismissing opposing viewpoints as hopelessly ignorant, biased, and dishonest.

First, let’s be clear about what Isenstein’s chart does show: a connection between “gun‐​related deaths” and certain gun laws. A simple statistical test offers decent evidence of such a relationship. But there’s a problem with using that evidence to conclude that more gun restrictions will reduce the number of murders or violent crimes. To understand why, consider the following story.

Suppose there are two towns, Chevyville and Fordburg. Many years ago, General Motors (GM) built a factory in Chevyville and its residents, in loyalty to their town’s largest employer, now all drive Chevrolets. In Fordburg, however, local leaders were so angry with GM for not building the plant there that they passed an ordinance banning Chevrolets.

Recently, some Chevyvillians noticed that every fatal car crash in town involved a Chevrolet, whereas Fordburg hasn’t had a fatal Chevy crash in years. Those folks assembled a damning chart comparing Chevyville and Fordburg’s Chevrolet‐​related fatal crashes and began demanding that Chevyville adopt Fordburg’s Chevrolet controls.

The Chevyvillians aren’t really upset about Chevrolet‐​related fatal crashes, but about fatal crashes in general. It’s quite possible that Chevy‐​free Fordburg has the same rate of fatal crashes as Chevyville, but Fordburg’s crashes involve Fords, Dodges, and Toyotas. That becomes obscured by Chevyvillians’ focus on Chevrolets. If Chevyville’s leaders, persuaded by the chart, were to ban Chevrolets, residents would likely continue suffering the same fatal crashes they do now, just with other car brands.

Isenstein (and many others) makes a similar analytical move as the Chevyvillians: she focuses on gun‐​related deaths rather than all violent deaths. Her table shows that, in states where it’s easier to buy and carry a gun, violent deaths are more likely to occur from guns—but that doesn’t mean the violent death rate is higher in those states.

Statistical testing

The question that lies at the heart of the gun control debate is whether gun restrictions reduce the incidence of murder and other violent crime. To help answer that, we can compare Isenstein’s state law data to murder and violent crime rates from the Bureau of Justice Statistics’ Uniform Crime Reports (UCR).

I did this with UCR data from 2012 (newer data aren’t yet available), resulting in 14 total test results (Isenstein’s seven laws multiplied by the two types of crime). The statistical test, known as a Pearson’s r, returns a value between 1 (indicating perfect positive correlation), 0 (no correlation), and –1 (perfect negative correlation). I arranged the tests so that negative numbers would indicate a relationship between more gun control and less violent crime.

The results: In half of the 14 tests, the resulting coefficients were positive numbers, meaning that Isenstein’s gun restrictions had no more than a coin‐​flip chance of yielding the results that gun control supporters expect. Moreover, in all 14 cases, the coefficients were tiny, with nine smaller than +/– 0.1. Those results should make us highly skeptical that the gun laws have any effect—positive or negative—on murder and violent crime.

I then slightly altered her law data in light of some peculiar data choices she made. For instance, Isenstein assumed that gun control changes implemented in 2015 affected gun‐​related deaths in 2013. She also treated states with court‐​established stand‐​your‐​ground (SYG) rights the same as states with no SYG, and different from states with legislatively enacted SYG rights. I used gun law data for 2012 (to match the crime data) and coded legislature‐​enacted SYG and court‐​imposed SYG the same.

I then repeated the statistical tests and the results were much the same as before. This time, nine of the 14 tests yielded positive numbers, indicating a correlation opposite of what gun control advocates expect. (In case you’re wondering about universal background checks, which President Obama is now considering expanding through executive order, both the murder and violent crime coefficients were positive numbers.) And again, all of the coefficients were tiny, with nine smaller than +/– 0.1.

Finally, mindful of the possibility that a combination of gun laws could produce a “compounding” effect that alters crime rates but is undetectable when the laws are considered separately, I created an overarching variable intended to distill all of Isenstein’s law data into a single “gun restrictive” measure for each state. For violent crime, the resulting coefficient was 0.093 (again, a positive number, contrary to gun control advocates’ expectation), while murder was –0.012 (a negative number, but extremely tiny even when compared to the other tiny coefficients).

The results weren’t all bad for gun control. I performed the same statistical tests on 2012 state suicide rates and found that five of the seven coefficients yielded negative numbers, which is what gun control supporters would expect. Two of the cases (handgun registry and open‐​carry) yielded coefficients as large as –0.267 and –0.254—still pretty small, but much bigger than the crime coefficients. So maybe gun control does have a small, beneficial effect on suicide. And it’s worth noting that nearly two‐​thirds of gun deaths are suicides. But currently the argument over gun control isn’t driven by concerns about suicide, but about violent crime.

Correlation, causation, and complexity

As damning as all this might appear, it shouldn’t be interpreted as proof that gun control (or gun access for that matter) has no effect on violent crime. My analysis suffers two serious shortcomings, which also plague Isenstein and her re‐​posters.

The first is the old dictum that correlation does not prove causation. Assume for a moment that our tests yielded stronger correlation coefficients; we still wouldn’t know whether they indicate that laxer (or tighter) gun controls led to higher murder and violent crime rates, or if the higher crime rates led to laxer (or tighter) gun controls as a result of public demand for different gun laws. Or some other cause could produce both higher murder rates and changes in gun laws.

Another problem is that many factors besides gun laws likely affect murder and violent crime rates (and suicides, for that matter—see Kessler’s and Volokh’s columns for more on this). Controlling for those factors could reveal different relationships (or non‐​relationships) than what Isenstein’s table and my analysis suggest.

Advanced statistical methods may be able to overcome those problems, and plenty of trees have been felled to provide paper for such academic analysis of American gun laws. Problem is, there’s hardly a consensus among researchers as to whether any causal relationship has been found. (For a sense of the literature, see this.) The United States is hardly alone in this. For instance, though gun control advocates are now lionizing Australia’s 1996 National Firearms Agreement restricting the licensing and ownership of different weapons, there appears to be no consensus among researchers as to whether the policy has reduced Australian homicide rates. (There does appear to be consensus that it has reduced suicide rates, though some researchers reach different conclusions.)

The difficulty with this research is that it’s hard to compare outcomes in the real world to outcomes in a hypothetical world where different gun laws exist but everything else is the same. Hence researchers’ use of advanced mathematics and statistics. The problem is, unless you have a Ph.D. in some statistical science and a good appreciation of the specific issues involved in gun research, you’ll be hard pressed to understand the critical points of that research, let alone form a knowledgeable opinion about which analyses are most likely correct.

That brings us to the derisive comments accompanying all those posts of Isenstein’s table. We now know the snark is misplaced. So why is the gun control debate, and American politics in general, so rife with such nastiness?

Part of the reason, understandably, is the stakes: gun control and gun rights involve some of the most cherished human values, including public safety, self‐​preservation, defense of innocents, privacy, and property rights. Part of the reason is simple fear: many people believe their risk of being victimized by violence is increasing (though the data show the opposite). And part of the reason is the trend in American politics over the last century: government has imposed itself so broadly that many issues are now winner‐​take‐​all, and people are desperate to avoid being on the losing side.

Those factors, along with the muddled complexity of gun research, should encourage more civil, open‐​minded, and respectful debate about gun laws, not to mention greater modesty about what policy can accomplish. Unfortunately, the opposite is happening. There’s a lot of Red team/​Blue team, “my side is smart and caring/​your side is stupid and cruel” bile.

Adding further fuel to this angry fire is the simple fact that people who dislike guns usually also dislike—and want to stick it to—people who like guns, and vice versa. And that’s a serious threat to American society, too.