This transcript was generated using AI automation and may contain minor formatting or transcription errors. Please refer to the original audio to verify specific quotes or context.

The Cato Podcast: How to Spot a Wrong Number

Host: Ryan Bourne, R. Evan Scharf Chair for the Public Understanding of Economics, Cato Institute Guest: Aaron Brown, author of Wrong Number: How to Extract Truth From a Blizzard of Quantitative Disinformation

Ryan Bourne: Statistics often enter policy debates not as evidence, but verdicts. Here in Washington, the words “study shows” can end an argument before it’s begun, especially when the finding is precise, dramatic, and politically useful. In recent years, we’ve been told that corporate profits drove inflation, that USAID spending prevented millions of deaths, and that a nationwide eviction moratorium could have prevented nearly two-fifths of America’s early COVID-19 mortality. Now, each of those claims strains credulity. Yet once a number appears in a prestigious journal, an official report, or a dramatic headline, it acquires authority and soon gets repeated as fact.

Aaron Brown’s new book, Wrong Number, is about what happens when that authority outruns the evidence. Some headline figures we hear are simply implausible. Others rest on bad definitions, cherry-picked periods, hidden assumptions, or correlations that are dressed up as causation. Sometimes the number is technically accurate, but the story attached to it is not. So why do so many influential claims travel from research papers into headlines, courtrooms, classrooms, and public policy when they fail basic tests of logic, rigor, and common sense?

Well, today I’m delighted to be joined by Aaron Brown himself on the podcast. Aaron has worked on hundreds of scientific projects as a data analyst, programmer, and statistician. He has also made money in arenas where bad quantitative judgment is punished pretty quickly: financial trading, poker, and other forms of gambling. So, Aaron, welcome to the Cato podcast.

Aaron Brown: Thank you for having me.

Ryan Bourne: So, Aaron, a wrong number is not always a made-up figure or an arithmetic mistake. So what makes a number wrong? What are you looking for when you decide to evaluate some of these things?

Aaron Brown: Well, I started — I was looking for things that were widely publicized in headlines. So I’m not looking for things that might corrupt the scientific field or cause misconceptions among experts. Plenty of other people look for those, and that’s valuable work too. But my focus was on things that go directly from researcher to media, maybe taking a brief stop at the university press relations office or something, and that people are intended to consume directly. Also, my wrong numbers are things that are obviously that — that at first glance are ridiculous, or with the briefest of scrutiny. One of my big ideas is that you don’t have to be a statistician. You don’t have to argue about the technical factors here. These are just numbers that are blatantly flawed, start to finish, and any reasonably skeptical person should be able to see it.

Ryan Bourne: So are there particular red flags that those of us who are interested in this should watch out for when we’re reading newspaper headlines? What are the things that get your spidey senses going when you see a headline?

Aaron Brown: Okay, the big one is direct to media, right? So this is something that comes directly from the researcher or the university. It isn’t the consensus of a field. It isn’t something that appears in a textbook. It isn’t a conference consensus opinion. Another one is it uses a lot of words that you don’t know the definitions of. It may sound like you know what they’re talking about. So when someone talks about how many people are in “near poverty” — well, you’ve got to go down and say, what is near poverty? Maybe is he telling me something I should know, or not? Or saying something is “dangerous” — well, what level? And so on. And a lack of context. Any number — anybody who’s trying to communicate information should be giving you some context for the numbers. Most of us don’t have it. So if you read that X causes some big number, is that a big number? Is it not a big number? That’s the kind of context you want. When you’re missing these things, that’s a good sign that you might be looking at a wrong number.

Something that seems like a revelation and happens to be politically useful, given where political debate is at any given time — certainly partisanship enters into this on both sides, and libertarians are as prone to this as anybody else. Libertarians have plenty of their own wrong numbers as well. So you can’t let your ideology be a shield.

Ryan Bourne: I want to come back to the libertarian wrong numbers perhaps a bit later, but let’s dig into some of the examples. So one example that you give in the book that’s really interesting was a Lancet paper claiming that USAID — that’s the U.S. International Aid Department — prevented precisely 39,663 deaths over a 20-year period. So before we touch on how they got to that number, why did the sheer scale of it set off your alarm bells?

Aaron Brown: I will answer that. I also want to say — here’s another red flag I forgot to mention. Numbers should be presented to their significance. You should only report the digits you have confidence in. So saying “about 90 million” is the right way to communicate that information. The authors did not count up all those deaths and have a list of the people they saved.

The problem here is the decline in global mortality over those 20 years was 79 million lives — about 79 million. I really should say about 80 million because I’m just using UN data — and that’s saying if the 2001 death rate had remained constant, 79 million fewer people would have died. So USAID is claiming to be responsible for 114% of the total decline in global mortality over that period. Now, USAID is about one-sixth of total global government foreign aid. Private foreign aid dwarfs government foreign aid, and lots of other things contributed: GDP growth, medical advances, and so forth. Then 60% of the decline in global mortality was in China, which received trivial amounts of USAID funding. And in fact, we’re pretty confident that the decline in mortality in China was due to their GDP increase, not due to anything USAID did. So there’s just no way this can be true.

And had the article presented any kind of context — they said that’s 114% of global decline in mortality — people would have realized, okay. But in fact, what the article did is they presented anti-context. They said, well, USAID spent $0.18 per U.S. taxpayer per day. Actually it’s a dollar — the authors aren’t really great on math. But either way, that context doesn’t tell you anything you can know: global deaths over 20 years versus one taxpayer over one day. That’s a way to obscure the number, to make it seem like anyone who opposes USAID valued pocket change more than millions of lives.

Ryan Bourne: So how on earth did their methodology end up giving USAID credit for essentially more than the whole global falling mortality? How on earth did they get to that number?

Aaron Brown: It’s actually pretty simple. Now, if you wanted to read through the paper, that’s where you need the PhD in statistics — it’s a very long paper, most of the important details are buried in an appendix, lots of jargon, lots of technical stuff. But the basic approach was very simple. They looked at USAID funding, which went up over the period. They looked at global mortality, which went down over the period. And they assumed — the methodology basically assumed — that all of the decline in global mortality was due to USAID increases, because they both got increased while the other decreased. But you could have used anything that increased over the period. Lots of things go up over time — they could have used my age, they could have used U.S. national debt, anything.

And worse — how did they get to 114%? That’s even worse: they excluded the five years where either USAID funding went down or global mortality went up. So five of the 21 years they threw out of the sample because they didn’t fit the story. And that’s how they got to 114%.

Ryan Bourne: So I think it sounds like, to use a term that economists like, they have an omitted variable bias problem in that methodology.

Aaron Brown: I would say that’s overly technical — if they had included all the other relevant variables. But nobody knows what all the variables are for global mortality. They did test 42 other variables and chose to include nine of them, which is another issue. You should preregister — you should say, here’s the ones we’re going to cast, not just pick and choose the nine that give you the best result.

Ryan Bourne: So the obvious risk is that some listeners will hear this and conclude, well, USAID maybe didn’t save that many. That’s not your argument. So how would we actually go about trying to judge whether aid programs were effective in saving lives, given all the data limitations that you’ve just indicated?

Aaron Brown: Sure. And in fact, there are people doing it. And one thing I want to emphasize here is wrong numbers — I’m not criticizing numbers that are just wrong. I’m criticizing numbers where people didn’t even try to get at the truth. There are a lot of researchers who are trying to do bottom-up estimates: how many mosquito nets didn’t get distributed because of the USAID cuts, how many antiviral drugs didn’t get distributed. And they’re coming up with some pretty big numbers here — coming up with numbers about a third the size of this 92 million. I personally consider even those numbers kind of soft. I suspect that if I were to look more carefully, I’d find that — well, I want to say certainly I would say it would shock me if USAID did not save millions of lives, and “million” is kind of pushing what I would consider plausible. But these people are really trying to get the right answer, and even though they’re coming in high, I would never criticize them as wrong numbers. But 92 million is just junk science.

Ryan Bourne: So there’s obviously a bit of a current controversy here because Elon Musk is getting a lot of heat. He was in an interview with The Economist last week where the economists were putting pressure on him about how USAID cuts influenced by DOGE might be sentencing to death millions of people, and California Representative Ro Khanna last month said that the USAID cuts in 2025 sentenced to death 4.5 million children. Does that pass your sniff test? Do you think that’s a kind of reasonable extrapolation?

Aaron Brown: Let’s get real here. We’re all under sentence of death, right? You’re all going to die. And until they invent immortality, no program could save lives — what the program can do is improve lives. It can make people live healthier, it can let them live longer, or something like that. So we’re already kind of dealing with a difficult conceptual problem here. Also, we only hear one side of the ledger.

Ryan Bourne: Sure.

Aaron Brown: If you give people money, or give them antiviral drugs, mosquito nets, fewer of them are going to die. But could that money be used more efficiently somewhere else? Could you just leave it in taxpayers’ pockets? When you take money from people, some of them die too — their mortality rates go up. So a useful public policy discussion needs good numbers. Another point here that gets lost is a lot of people oppose USAID not because it doesn’t save lives, but because they feel it entrenches a lot of corrupt governments. And if USAID allows a dictator to stay in place, even if it saves a million lives in a country, that dictator might cost 10 million lives or ruin 10 or 20 million lives. So you really have to look at the whole picture, and looking at the whole picture requires good numbers for each piece of it. The 92 million just short-circuits everything.

Ryan Bourne: Yeah. And I think it would be churlish to suggest that just stripping away funding all of a sudden might not condemn certain people to certain illnesses and conditions that they otherwise wouldn’t have got. But then I thought Elon Musk made an important point, that in the longer run, of course, there’s every reason to expect that some non-profits, some charities, to the extent that there’s real wants and needs there, would step in to provide some of these programs. So I think that’s going to be a debate that rumbles on.

You are open about your libertarian priors — you’re open about that in the book. Do you think that affects which claims catch your eye, or do you consider yourself willing to do the hard yards to assess studies across the board?

Aaron Brown: Well, I try. I’m sure that there is some bias there. So if I run across a study that says lowering marginal tax rates improves the economy, I’d probably drill down as hard as I would on one that said a wealth tax is going to double the economy. But I do try. The thing is, the wrong numbers I get are ones that are blown up by media, and media actually typically takes a pretty skeptical eye toward libertarian claims. So the kind of studies that come up — I mention this in the book — it sounds like it has a populist bias, because the wrong numbers are things that are promoted by experts and questioned by populists. So a lot of these claims are things that get published because they have experts behind them. Libertarians are the most skeptical. There are some sources — the Wall Street Journal, and some others — that will even give some weight toward free market arguments, freedom arguments, things like that. But they certainly don’t get picked up with the lack of skepticism that you get from certain kinds of progressive ideas or expert ideas.

Ryan Bourne: So do you think most of the wrong numbers that you assess get blown up because of the marketing and media interpretation, or is the underlying analysis just not rigorous? I know that every wrong number is kind of different, and I’m not saying that these things are mutually exclusive, but are the bulk of the wrong numbers things that you think have just been marketed in a certain way, told a dodgy story, or is it the actual analysis that’s the problem?

Aaron Brown: Well, there are some examples in the book of good studies that got misreported, but most of the examples were not good studies. And when I say not good, I don’t mean I found some technical, statistical things I’d do differently — I mean that there is no possibility that these people could have found the truth the way they did. Either they didn’t have the data, or the methodology just could not get at the truth. And it doesn’t surprise me that people send some of these things directly to the media — they don’t get much traction in the scientific literature. The other scientists look at this and understand that this is not a good number, but the media takes it and prints it.

What I really don’t understand, and what I think is the really key issue here, is why don’t they go back later and correct it? A lot of these statistics have been completely discredited — reporters were lied to, and you would think reporters would try and go back: how did this happen, who lied to us, what should we do? But they don’t. They just forget it and go on with the next story.

The only answer I have for this — David Zweig wrote a book, An Abundance of Caution, about school closures during COVID, which is a deep dive on one particular wrong number — and his answer, which is the best I’ve seen, is tribalism. People take these numbers as tribal badges, and they’re no more meaningful as numbers or statistics than what tattoo you put on your face for some prehistoric tribe. And once that tattoo’s on the forehead, it doesn’t matter where it came from or what it means.

Ryan Bourne: I’m glad you mentioned the pandemic, because I think that was a particularly egregious period of a lot of dodgy statistics and studies flying around, I think probably given the high stakes. So people were purporting that certain policies could save tens of thousands of lives, really help alleviate the damage of the pandemic. There was a lot of bad reporting on epidemiological studies, some of which suggested what would happen in terms of number of deaths if we did nothing in reaction to the pandemic — and of course, that was never a realistic possibility, because the existence of the virus meant we were all going to change our lives voluntarily in certain ways.

But one Duke paper that you hone in on said that a universal eviction moratorium could have prevented over 40% of U.S. COVID-19 deaths over eight months. So how on earth do they get to that? Because to my mind, evicting certain people from a property and other people moving in — yes, you can imagine certain people then going into a shelter or whatever, but it seems to me that’s unlikely to be a big population. So how on earth could that lead to 40% of COVID-19 deaths being avoided? How did they get to that sort of estimate?

Aaron Brown: Yeah, it’s a good example that has every red flag. So the first thing is, their argument was — and this was the argument that was used to justify both the federal and many state and local eviction moratoriums that were used in court, it was read into the Congressional Record, so this is a paper doing a lot of serious policy work — their argument was, well, if you evict somebody, they might move in with other people to increase crowding, and increased crowding increases infection. But a second thought tells you crowding is number of renters divided by number of residences, and evicting somebody doesn’t destroy their apartment — somebody else moves in. And in fact, eviction moratoriums, if landlords cannot evict people for nonpayment of rent, mean they build fewer units, convert fewer units, remove units from the market. So the policy is doing exactly the opposite of what it’s claimed.

Then there’s the fact that lots of people study evictions, and they would notice that there’s nobody to study, because all the evicted people are dead. I mean, the 40% of COVID deaths literally meant 100% of everybody evicted had to die of COVID. And people who studied COVID deaths would notice — hey, not that many people are actually evicted. There are a lot of forced moves — this is something that’s often missed. There’s something like seven forced moves to every eviction.

Ryan Bourne: So when you say forced move, do you mean like a landlord increases rent significantly and so a family kind of feels like they have to move out — is that a forced move?

Aaron Brown: The main thing I mean is the landlord comes up and says get out or I’m going to evict you, and so you move — you don’t want to go through the court proceeding and all that. Most people do move. Some of it is, as you say, forced by raising rent. Some of it is forced by illegal means — the landlord just takes the door off. So these things do happen, but evictions themselves are actually pretty rare, and they were very low during COVID. A lot of courts were closed, so you couldn’t do it. A lot of judges simply weren’t ordering eviction. So it was an extraordinary statistic that couldn’t be true, on top of which nobody’s got data for it. The authors claim they had county-level data on evictions, which they needed for their study, but the source they gave for it was a source that didn’t have county-level data. Most of their other sources they gave no data source at all. This was a working paper — it was not peer-reviewed, published.

And we wrote to the Duke unit, we wrote to the researchers first, and they refused to give the data. We went to the University Office of Scientific Integrity, which Duke University set up because they’ve had so much problem with covering up bad research and lost a lot of money in lawsuits — they never got back to us. We went to NBER, the National Bureau of Economic Research, that published the paper — they just claimed responsibility, they said we don’t publish it, we just let the researchers print it up and encourage them to share their data, but we don’t require it. So nobody knows where the data from this paper came from, and frankly I’m skeptical that the data even exists — I’ve never been able to find it. So there’s every red flag on this paper, yet not only did it sail through and cause major policy changes, but nobody has since gone back to question it.

Ryan Bourne: And by the way, I don’t know whether they submitted it for peer review or not, but it has not appeared as a peer-reviewed publication.

Aaron Brown: Peer review — I’m not a big fan of peer review. I think peer review should happen after publication. You publish it and let people criticize and figure out the truth — you don’t suppress things ahead of time. But in that case, peer review would certainly have required them to show their data, if nothing else.

Ryan Bourne: Yeah, and I know there are a few journals coming about now where the whole purpose of the journal is to try and replicate things after publication. That seems to me a valuable exercise. But I just wondered, when you’re assessing some of these wrong numbers, did you find any academic disciplines that you think are better at avoiding them — or is this a widespread problem across economics, sociology, and everything else?

Aaron Brown: There are definitely differences. There are fields that exercise pretty good control, and they tend to be the fields that are less partisan. But you have to be careful here — for example, this Duke University paper was a combination of economics and public health. Now, if you read something in the Journal of Economic Theory, in the top economics journals, most of the time it is pretty good. But on the medical research side — Lancet is the most prestigious medical journal in the world, the oldest major one still around — they have this long list of all the incredible things they do to rigorously test their papers, they seek out people with contrary views, they run a dice hitting professionals — and yet they probably publish some of the biggest junk around. Psychology, especially behavioral psychology, has a terrible reputation of not only terrible research but fraud. Physics is pretty good — you occasionally get the bad paper in physics, but when you do, people don’t just forget about it, it’s not put under the rug — there are people who write books about the scandal and look hard at how to prevent it going forward. Nutrition science — anything that gives you a clickbait headline — yeah, you’re going to find a lot of bad research.

Ryan Bourne: Yeah, I find some of the worst of the economics research actually comes out of international agencies. I ran into an example, pandemic-related, with a paper endorsed by Nobel Prize winner Joseph Stiglitz from the UN, claiming that if the US had had lower income inequality — think like New Zealand-level income inequality — for the first couple of years of the pandemic, then we’d have avoided 160,000 pandemic deaths. There was no correlation between inequality by country and excess deaths by country in the raw data — it was the control variables doing all of the work, which raised red flags for me. And then I tried replicating it over the full period of the pandemic, and there was absolutely no relationship at all between the variables. So I kind of see this all the time, and I think there’s a tendency when you see a study like that to think there’s motivated reasoning here.

And you say in conclusion of the book that wrong numbers, like appeals to authority, are not intended for ordinary people to understand or to help them make decisions — they’re intended to get ordinary people to accept the wishes, or I guess the wisdom, in this case, of the expert. So why do you think numbers so often blind people to logical thinking, to giving a strong degree of scrutiny? Why do you think bad arguments expressed through numbers seem to blind us in a way that arguments by philosophy or logical reasoning don’t?

Aaron Brown: An excellent question. You might ask the same thing 500 years ago — why was the fact that somebody found a dusty text that said something more important than common sense or experiment? The number carries with it an authority, I think, that’s inherited from physics, from science. Five hundred years ago, if somebody had said the earth is 93 million miles from the sun, or it takes light eight minutes to go from the sun to the Earth, nobody would have taken them seriously. But now we take those numbers and we trust them. And so people bring out other numbers that seem like they’re as well-defined, as carefully validated as physics numbers, and they inherit some of the authority that comes from that.

Ryan Bourne: Sounds like you know exactly what you’re talking about — there’s a calculation behind it, there’s data behind it. Often there isn’t. Sometimes you click on the “studies prove” link and there’s no studies. Sometimes there are studies and they say something totally different. Sometimes there are studies that say what the quote said, but they’re based on terrible data.

Aaron Brown: That’s another question you can ask: could anybody possibly have the data to answer this question? Could anybody possibly know how U.S. income inequality affected pandemic deaths? I can’t think of any possible way you could get data to answer that question objectively.

Ryan Bourne: Yeah, no, I totally agree with that. Another area that our interests overlap is on the issue of supposed greed — corporate greed driving inflation. Now, there was a study, if you could call it that, that went viral a few years ago by the left-wing campaign group Groundwork Collaborative, and they used national income data and compared that to forms of inflation to try to claim that corporate profits were responsible for, or driving, some of the post-pandemic inflation. So talk to us about that one — what were they doing, and why was that a wrong number?

Aaron Brown: Well, mostly what they were doing was cherry-picking. They picked periods of time where corporate profits were going up and inflation was going up, but wages weren’t — or weren’t going up as fast — and you can certainly find periods like that. If you expand the window, what you find is that the opposite had been true for two or three years — wages had been going up faster, inflation had been low, and corporate profits had been going down, and then this was sort of catch-up. So when you look at the whole period, there doesn’t seem to be much there.

Another major problem in the paper is they misunderstood the producer price index. They took the producer price index as the cost of making goods, and they said, well, look — the consumer price index is going up faster than the producer price index, so prices are going up faster than costs, and companies are stealing the difference. But the producer price index is a price index — it’s the price that goods are sold wholesale. When the producer sells the goods, what do they get? So this was actually evidence that the producers were making less money, right — not that they were stealing from consumers.

Now, again, when you look at a longer period, you find out that the producer price index and the consumer price index really move almost completely in tandem over the medium term, but over a six-month or one-year period they can diverge sometimes — prices to consumers go up faster than prices to the wholesalers, from the wholesalers to producers, and sometimes the reverse is true. And most fundamentally, we know a lot — well, we don’t know a lot about economics, but we have pretty good models of what causes inflation, and we have zero models for corporate greed or what it is. And there’s no particular reason to think that corporate greed has changed dramatically in the last year or two. We are seeing rising corporate profits — medium to long term — and there are reasons for it, and there are people who might legitimately say we should slow that down, we should increase corporate taxes. I don’t personally agree, but these are rational arguments you can have. But labeling it “greed” is almost never a useful thing — it means somebody acted in their own interests instead of yours, and that doesn’t help much in policy debates.

Ryan Bourne: Yeah. No, I thought this one was particularly egregious when I came across it at the time. As you say, they were comparing a two-quarter period in 2023, a period when actually prices didn’t go up that much, but because profits had gone up as a large proportion of that change in prices, they were able to suggest that profits were driving a big share. I think the more fundamental problem is that this is kind of economics by accounting identity. Yeah, you can disaggregate a selling price into labor costs, other costs, profit per unit — but when you get something like a whole lot of money printing that is surging demand across the economy, that can temporarily lift prices and profits together, before other costs adjust. So there are economic explanations for why factors like the bad policy that we lived through drove an increase in prices and profits in certain industries. But of course, as you say, this was useful because Groundwork Collaborative and many of the progressive groups were pushing for windfall taxes, pushing for antitrust action against big business and for price controls, and so emphasizing the role, as they saw it, of profits in causing inflation took a lot of the focus away from other potential solutions to the inflation problem, like choking off fiscal and monetary excess.

So let’s move on to issues of how much we should trust experts. Most of the book, I think it’s fair to say, isn’t about outright fraud or misrepresentation. But what’s your honest assessment — because obviously we’re talking about specific wrong numbers here, we’re not talking about the full range of studies that are published in every field — what’s your best estimate of the baseline level of accuracy in academic research? You’re picking particularly egregious examples of numbers that are clearly wrong — I guess we shouldn’t extrapolate from that to say academic research is plagued by dishonesty — but how much of a problem is it within academic research?

Aaron Brown: It absolutely depends on your denominator — which you look at. Here’s the fundamental problem with academic research: almost nobody can produce the amount of high-level publications that university promotion committees and grant-granting agencies demand. So virtually every researcher has to play some tricks — they have to do some low-quality studies. Now, those aren’t necessarily wrong numbers, they can just be kind of lightweight studies — you send a questionnaire to your sophomore psychology class and you publish the results as a paper. That doesn’t really tell you very much about anything, but it’s not false. Maybe there’s something interesting there, or you do some wrong numbers or something like that.

But that doesn’t destroy most fields, because people know what’s the good and the bad research. If I do a lightweight paper to get what I need for tenure, and I do a really good, serious piece of research that I care about, the good research I put around, I send it to people, I talk it up at conferences, I push it in lots of ways, and it gets a lot of attention. The other paper — I never mention it, nobody ever reads it, nobody ever cares, it doesn’t do much good, unless it gets picked up by some sort of media amplification for partisan reasons.

So if you just take the assumption that just because something appeared in a peer-reviewed journal — all that tells you is somebody needed a publication and had enough friends among the peer reviewers who were doing the same game themselves, or they were somehow in the network so they could get it published. So just the fact that it’s published doesn’t tell you much. But if you look at the publications in the top journals — not in most fields, unfortunately, but in most fields — and if you look at the textbooks in particular, that’s what I kind of say: if it’s in a textbook, unless it’s a terrible field, it’s probably worth considering, because it tends to really have some consensus among the experts.

Ryan Bourne: So I want to hear the wrong numbers that libertarians are guilty of appealing to.

Aaron Brown: All right. Well, a few of them come from — I was at the University of Chicago, I took courses from Sam Peltzman, who was a brilliant guy who did some really great work about how the FDA kills 100 people for every one person it saves, and how mandatory seatbelt laws kill people. Now, these were very careful papers — there’s a lot of nuance to those conclusions, there’s been a lot of research since. But I would say something on the order of half the libertarians I run into remember the cartoon version of what Sam said back in the 1970s and haven’t updated their opinions on it since. It’s just too perfect — it fits too closely into the libertarian prejudices. The FDA has changed enormously in that time, and like I say, a lot of new research has come out, and the original conclusion had a lot of nuances and qualifications in it.

I have actually one chapter in my book about whether Medicaid saves lives, and I think it was back in 2014, there was a famous Oregon study that claimed Medicaid saves zero lives. And then there’s a more recent paper that came out that said Medicaid saves millions of lives. Both of these were good papers with a lot of nuance — they don’t actually disagree by that much — but I would say most libertarians I know would say, yeah, the Oregon study was carved in stone, brought down from Mount Sinai, and this latest one is just some trashy progressive research put out. In fact, both are good papers, both are worth reading, and whether or not Medicaid saves lives or at least extends lives is absolutely open to debate.

Ryan Bourne: I remember a few years ago I hosted Jason Furman, the center-left economist who used to work at the Council of Economic Advisers for President Obama — I think he was also chief economist for a while — and he gave a speech in which he said progressives are often guilty of getting the sign wrong: they assume a causal effect runs in one direction, but actually often get it wrong and it’s a causal effect in the other direction. Like, subsidizing green energy is good for growth and good for reducing carbon emissions, when actually it’s bad for growth in the short term, because you’re subsidizing otherwise uneconomic forms of electricity generation. But then he said free-market conservatives tend to get the magnitudes wrong — they tend to identify a real-world unintended consequence of a policy, but then often assume that is the major effect and that it outweighs all of the other trade-offs entailed with the policy. So it sounds like your critique of the numbers libertarians get wrong kind of fits into that framework pretty well.

Aaron Brown: Yeah, I think that’s true. I don’t get libertarians to be logical people — they have logical beliefs, but they’re not always really great on the quantitative level. The philosophy does sort of lend itself to absolutes — it’s either good or bad, it’s either freedom or tyranny, or something like that — whereas there are an awful lot of progressive ideas: greed, inflation, the eviction moratorium — things that are just illogical on the face, and a lot of numbers, but big numbers. But the logic just escapes me.

Ryan Bourne: Well, I’m afraid that’s all we’ve got time for. But I think the way that you describe how a sound study is like a brick that has to fit into a larger structure of knowledge is really important, and I think that’s a really good way for non-experts to think about expertise — that we should trust mature bodies of work more than prestigious, flashy one-off studies. And I think the central takeaway of this discussion is that a statistic should begin a conversation and not end it. We often have to ask basic questions like: what is being counted, compared with what? Is this scale actually plausible? Does the timing fit? Is the result actually observed, or is it something that’s chucked out from a model? The aim, I think, for us should be to demand evidence that is strong enough for the claim, and to prefer cumulative and contestable knowledge to these one-off results.

So I really appreciate all the work you’ve done on this, and some of the fantastic videos that you’ve done in line with this body of work for our friends at Reason — do check them out on YouTube, they’re very good fun. So thank you for listening to today’s episode of the Cato Podcast. Once again, I’m Ryan Bourne, joined by Aaron Brown. If you enjoyed today’s discussion, please subscribe and leave a review wherever you get your podcasts. To read Aaron Brown’s book Wrong Number, you can check it out online — I’m sure it’s on Amazon’s bookstore. And to learn more about the ideas discussed in this episode, you can visit cato​.org.

The Cato Podcast is a production of the Cato Institute, dedicated to advancing individual liberty, limited government, free markets, and peace. Join us next time for more insights and conversations on the issues shaping our world.