For years, scientists and non-scientists alike have complained that something is fundamentally wrong with the way we do this business. Something has corrupted the integrity of our science.
This is a serious charge because it means that more and more government policy — from limiting carcinogens to regulations on carbon emissions — is based upon an increasingly polluted canon of knowledge. If that were somehow corrected for, we would live under a far less intrusive government.
Last week, this view received strong support when two researchers, Paul Smaldino and Richard McElreath, published a bombshell article in a journal of Britain’s Royal Society called “The Natural Selection of Bad Science.” Put simply, it is a closely argued, mathematically rigorous demonstration that the way we now reward scientists is actually making science worse.
The things that scientists crave — like tenure and research funding — incentivize frequent publishing of massive numbers of academic papers. To publish that much, you need a tremendous amount of financial support. And when it comes to scientific work that could have regulatory implications, almost all of the money comes from Washington.
We are not suddenly becoming more intelligent and getting everything right. What’s happening is that scientists are responding to incentives.
As Smaldino and McElreath explain in their study, this rush for the printing presses leads to sloppy science and declining standards of rigor. So, by extension, the more money the government throws at some field with an initially limited number of practitioners (think global warming) the worse the science will become.
What constitutes “bad science”? It’s the epidemic of positive results, in which a researcher reports that the data support his or her prior hypothesis. Stanford’s Daniele Fanelli has shown a distressing increase of positive results in recent decades, something that can’t be true in the real world. Think about it — we are not suddenly becoming more intelligent and getting everything right. What’s happening is that scientists are responding to incentives.
Usually, hypotheses are put forward in some grant proposal. Financial backers don’t like negative findings, because negative findings don’t support the work that they’ve funded. Supervisors lose face and researchers can lose their funding.
There’s an additional wrinkle on this that neither the authors nor anyone else has discussed. What happens when the government massively funds something that really isn’t science?
By “science” I mean “hypotheses that can be subjected to stringent tests.” The philosopher of science Karl Popper said science that couldn’t be tested is really just “pseudoscience.” Popper criticized philosophies claiming the scientific mantle that are used to explain pretty much everything.
His favorites were psychoanalysis and Marxism. If he were alive today he would see parallels when prominent climatologists explain pretty much every and any weather anomaly — a big rainstorm, a big drought, lack of snow, or a big blizzard — as “consistent with” the effects of global warming. It’s a good bet that climate science, which is primarily the generation of unverifiable prospective models (after all, the future isn’t here yet) would have made Popper’s list.
So, instead of being rewarded for research that supports a prior hypothesis, no matter how sloppy it is, those involved in climate studies get published a lot not by testing (which can’t be done in the prospective sense) but by producing dire, horrific results. Because these often appear in prominent journals — which love to feature articles that generate big news stories — the greater the horror, the more likely is promotion, citation and more money.
This then generates more and more of these perverse incentives in a vicious cycle.
All of this is well and good and could be dismissed as just another example of how incentives drive supposedly dispassionate scientists. But in several fields, like climate, the accumulation of horrific literature is often summarized by governments, usually to support some policy. Bad science then justifies bad policy.
It is quite significant that Smaldino and McElreath’s paper was published by the Royal Society. Surely they know the result will be more distrust of the modern scientific enterprise, and, by extension, in the policies supported by it. The fact of its publication is evidence that we have reached a turning point, where the pollution of modern science is now an accepted truth.