Tag: data

Is There No “Hiatus” in Global Warming After All?

A new paper posted today on ScienceXpress (from Science magazine), by Thomas Karl, Director of NOAA’s Climate Data Center, and several co-authors[1], that seeks to disprove the “hiatus” in global warming prompts many serious scientific questions.

The main claim[2] by the authors that they have uncovered a significant recent warming trend is dubious. The significance level they report on their findings (.10) is hardly normative, and the use of it should prompt members of the scientific community to question the reasoning behind the use of such a lax standard.

In addition, the authors’ treatment of buoy sea-surface temperature (SST) data was guaranteed to create a warming trend. The data were adjusted upward by 0.12°C to make them “homogeneous” with the longer-running temperature records taken from engine intake channels in marine vessels. 

As has been acknowledged by numerous scientists, the engine intake data are clearly contaminated by heat conduction from the engine itself, and as such, never intended for scientific use. On the other hand, environmental monitoring is the specific purpose of the buoys. Adjusting good data upward to match bad data seems questionable, and the fact that the buoy network becomes increasingly dense in the last two decades means that this adjustment must put a warming trend in the data.

The extension of high-latitude arctic land data over the Arctic Ocean is also questionable. Much of the Arctic Ocean is ice-covered even in high summer, meaning the surface temperature must remain near freezing. Extending land data out into the ocean will obviously induce substantially exaggerated temperatures.

Additionally, there exist multiple measures of bulk lower atmosphere temperature independent from surface measurements which indicate the existence of a “hiatus”[3]. If the Karl et al., result were in fact robust, it could only mean that the disparity between surface and mid-tropospheric temperatures is even larger that previously noted. 

Getting the vertical distribution of temperature wrong invalidates virtually every forecast of sensible weather made by a climate model, as much of that weather (including rainfall) is determined in large part by the vertical structure of the atmosphere.

Instead, it would seem more logical to seriously question the Karl et al. result in light of the fact that, compared to those bulk temperatures, it is an outlier, showing a recent warming trend that is not in line with these other global records.

And finally, even presuming all the adjustments applied by the authors ultimately prove to be accurate, the temperature trend reported during the “hiatus” period (1998-2014), remains significantly below (using Karl et al.’s measure of significance) the mean trend projected by the collection of climate models used in the most recent report from the United Nation’s Intergovernmental Panel on Climate Change (IPCC). 

It is important to recognize that the central issue of human-caused climate change is not a question of whether it is warming or not, but rather a question of how much. And to this relevant question, the answer has been, and remains, that the warming is taking place at a much slower rate than is being projected.

The distribution of trends of the projected global average surface temperature for the period 1998-2014 from 108 climate model runs used in the latest report of the U.N.’s Intergovernmental Panel on Climate Change (IPCC)(blue bars). The models were run with historical climate forcings through 2005 and extended to 2014 with the RCP4.5 emissions scenario. The surface temperature trend over the same period, as reported by Karl et al. (2015, is included in red. It falls at the 2.4th percentile of the model distribution and indicates a value that is (statistically) significantly below the model mean projection.

The distribution of trends of the projected global average surface temperature for the period 1998-2014 from 108 climate model runs used in the latest report of the U.N.’s Intergovernmental Panel on Climate Change (IPCC)(blue bars). The models were run with historical climate forcings through 2005 and extended to 2014 with the RCP4.5 emissions scenario. The surface temperature trend over the same period, as reported by Karl et al. (2015, is included in red. It falls at the 2.4th percentile of the model distribution and indicates a value that is (statistically) significantly below the model mean projection.


[1] Karl, T. R., et al., Possible artifacts of data biases in the recent global surface warming hiatus. Scienceexpress, embargoed until 1400 EDT June 4, 2015.

[2] “It is also noteworthy that the new global trends are statistically significant and positive at the 0.10 significance level for 1998-2012…”

[3] Both the UAH and RSS satellite records are now in their 21st year without a significant trend, for example

In Holding NSA Spying Illegal, the Second Circuit Treats Data as Property

The U.S. Court of Appeals for the Second Circuit has ruled that section 215 of the USA-PATRIOT Act never authorized the National Security Agency’s collection of all Americans’ phone calling records. It’s pleasing to see the opinion parallel arguments that Randy Barnett and I put forward over the last couple of years.

Two points from different parts of the opinion can help structure our thinking about constitutional protection for communications data and other digital information. Data is property, which can be unconstitutionally seized.

As cases like this often do, the decision spends much time on niceties like standing to sue. In that discussion—finding that the ACLU indeed has legal standing to challenge government collection of its calling data—the court parried the government’s argument that the ACLU suffers no offense until its data is searched.

“The Fourth Amendment protects against unreasonable searches and seizures,” the court emphasized. Data is a thing that can be owned, and when the government takes someone’s data, it is seized.

In this situation, the data is owned jointly by telecommunications companies and their customers. The companies hold it subject to obligations they owe their customers limiting what they can do with it. Think of covenants that run with land. These covenants run with data for the benefit of the customer.

Will the Administration Make a Run at Transparency?

Last fall, I reported that the Obama administration lagged the House of Representatives on transparency. The conclusion was driven by a study of the quality of data publication regarding key elements of budgeting, appropriating, spending, and the legislative process. (Along with monitoring progress in these area, we’ve been producing data to show that it can be done, to produce a cadre of users, and to simply deliver government transparency at a less plodding pace.)

There are signs that the administration may make a run at improving its transparency record. Buried deep in the FY 2014 budget justification for the Treasury Department’s Bureau of the Fiscal Service, it says that funds will support “government-wide data standardization efforts to increase accuracy and transparency of Federal financial reporting.” That means the public may get better access to where the money goes – outlays – in formats that permit computer-aided oversight.

In parallel, a Performance.gov effort called the Federal Program Inventory says that, in May of 2014, it will publish a Unique Federal Program Inventory number (pg. 4-5) for each federal program, along with agency IDs and bureau IDs. This may be the machine-readable federal government organization chart whose non-existence I have lamented for some time.

If this sounds jargon-y, you’re normal. Think of federal spending as happening on a remote jungle island, where all the inhabitants speak their own language. On Federal Spending Island, no visitor from the U.S. mainland can understand where things are there, or who is saying what to whom.

True machine-readable data will turn Federal Spending Island into a place where English is spoken, or at least a some kind of Federal Spending-English dialect that makes the movement of our tax dollars easier to track.

The Data Says Open-Ended Spending Bills Are Common

Let’s start with a little civics lesson: Congress spends money through a two-step process. Spending must first be authorized. That’s called an authorization of appropriations. Then, in a second step, the money is actually appropriated. There are exceptions, but on the whole this is how spending works. Authorizing bills go to authorizing committees, and appropriations bills go to the appropriations committees. When both do their thing, money gets spent. It’s good to keep an eye on.

In our project to generate better data about what Congress is doing, we’ve “marked up” over 80 percent of the bills introduced in Congress so far this year, adding richer and more revealing computer-readable data to the text of bills. That’s over 4,000 of the 5,000-plus bills introduced in Congress since January. We’re to the point where we can learn things.

I was surprised to find just how often the bills that authorize spending leave the amounts open-ended. A recent sample of the bills we’ve marked up includes 428 bills with authorizations of appropriations. Just over 40 percent of them place no limit on how much money will be spent. They say things like “such sums as may be necessary,” leaving entirely to the appropriations committees how much to spend. (There are many bills with both defined amounts and open-ended spending. To be conservative, we treated any bill having limited spending as not unlimited.)

This leads me to two related conclusions. First, authorizations of appropriations being a potential brake on spending, this surprisingly common practice is part of Congress’s fiscal indiscipline. The members of Congress and Senators who introduce such bills and vote to authorize open-ended spending are avoiding their responsibility to determine how much a program is worth to us, the taxpayers.

Cato’s “Deepbills” Project Advances Government Transparency

It’s not the culmination–that will come soon–but a major step in our work to improve government transparency has been achieved. I’ll be announcing and extolling it Wednesday at the House Administration Committee’s Legislative Data and Transparency conference. Here’s a quick survey of what we’ve been doing and the results we see on the near horizon.

After president Obama’s election in 2008, we recognized transparency as a bipartisan and pan-ideological goal at an event entitled: “Just Give Us the Data.” Widespread agreement and cooperation on transparency has held. But by the mid-point of the president’s first term, the deep-running change most people expected was not materializing, and it still has not. So I began working more assiduously on what transparency is and what delivers it.

In “Publication Practices for Transparent Government” (Sept. 2011), I articulated ways the government should deliver information so that it can be absorbed by the public through the intermediary of web sites, apps, information services, and so on. We graded the quality of government data publication in the aptly named November 2012 paper: “Grading the Government’s Data Publication Practices.”

But there’s no sense in sitting around waiting for things to improve. Given the incentives, transparency is something that we will have to force on government. We won’t receive it like a gift.

So with software we acquired and modified for the purpose, we’ve been adding data to the bills in Congress, making it possible to learn automatically more of what they do. The bills published by the Government Printing Office have data about who introduced them and the committees to which they were referred. We are adding data that reflects:

- What agencies and bureaus the bills in Congress affect;

- What laws the bills in Congress effect: by popular name, U.S. Code section, Statutes at Large citation, and more;

- What budget authorities bills include, the amount of this proposed spending, its purpose, and the fiscal year(s).

We are capturing proposed new bureaus and programs, proposed new sections of existing law, and other subtleties in legislation. Our “Deepbills” project is documented at cato.org/resources/data.

This data can tell a more complete story of what is happening in Congress. Given the right Web site, app, or information service, you will be able to tell who proposed to spend your taxpayer dollars and in what amounts. You’ll be able to tell how your member of Congress and senators voted on each one. You might even find out about votes you care about before they happen!

Having introduced ourselves to the community in March, we’re beginning to help disseminate legislative information and data on Wikipedia.

The uses of the data are limited only by the imagination of the people building things with it. The data will make it easier to draw links between campaign contributions and legislative activity, for example. People will be able to automatically monitor ALL the bills that affect laws or agencies they are interested in. The behavior of legislators will be more clear to more people. Knowing what happens in Washington will be less the province of an exclusive club of lobbyists and congressional staff.

In no sense will this work make the government entirely transparent, but by adding data sets to what’s available about government deliberations, management and results, we’re multiplying the stories that the data can tell and beginning to lift the fog that allows Washington, D.C. to work the way it does–or, more accurately, to fail the way it does.

At this point, data curator Molly Bohmer and Cato interns Michelle Newby and Ryan Mosely have marked up 75% of the bills introduced in Congress so far. As we fine-tune our processes, we expect essentially to stay current with Congress, making timely public oversight of government easier.

This is not the culmination of the work. We now require people to build things with the data–the Web sites, apps, and information services that can deliver transparency to your door. I’ll be promoting our work at Wednesday’s conference and in various forums over the coming weeks and months. Watch for government transparency to improve when coders get a hold of the data and build the tools and toys that deliver this information to the public in accessible ways.

Three Lessons from the Increasingly Irrelevant Annual Wiretap Report

The 2011 Wiretap Report was released this weekend, providing an overview of how federal and state governments used wiretapping powers in criminal investigations. (Surveillance for intelligence purposes is covered in a separate, far less informative report.) There’s plenty of interesting detail, but here’s the bottom line:

After climbing 34 percent in 2010 the number of federal and state wiretaps reported in 2011 deceased 14 percent. A total of 2,732 wiretaps were reported as authorized in 2011, with 792 authorized by federal judges and 1,940 authorized by state judges…. Compared to the numbers approved during 2010 the number of applications reported as approved by federal judges declined 34 percent in 2011, and the number of applications approved by state judges fell 2 percent. The reduction in wiretaps resulted primarily from a drop in applications for narcotics.

So is the government really spying on us less? Is the drug war cooling off? Well, no, that’s lesson number one: Government surveillance is now almost entirely off the books.

The trouble, as Andy Greenberg of Forbes explains, is that we’ve got analog reporting requirements in a digital age. The courts have to keep a tally of how often they approve traditional intercepts that are primarily used to pick up realtime phone conversationse—96 percent of all wiretap orders. But phone conversations represent an ever-dwindling proportion of modern communication, and police almost never use a traditional wiretap order to pick up digital conversations in realtime. Why would they? Realtime wiretap orders require jumping all sorts of legal hurdles that don’t apply to court orders for stored data, which is more convenient anyway, since it enables investigators to get a whole array of data, often spanning weeks or month, all at once. But nobody is required to compile data on those types of information requests, even though they’re often at least as intrusive as traditional wiretaps.

From what information we do have, however, it seems clear that phone taps are small beer compared to other forms of modern surveillance. As Greenberg notes, Verizon reported fielding more than 88,000 requests for data in 2006 alone. These would have ranged from traditional wiretaps, to demands for stored text messages and photos, to “pen registers” revealing a target’s calling patterns, to location tracking orders, to simple requests for a subscriber’s address or billing information. Google, which is virtually unique among major Internet services in voluntarily disclosing this sort of information, fielded 12,271 government requests for data, and complied with 11,412 of them. In other words, just one large company reports far more demands for user information than all the wiretaps issued last year combined. And again, that is without even factoring in the vast amount of intelligence surveillance that occurs each year: the thousands of FISA wiretaps, the tens of thousands of National Security Letters (which Google is forbidden to include in its public count) and the uncountably vast quantities of data vacuumed up by the NSA. At what point does the wiretap report, with its minuscule piece of the larger surveillance picture, just become a ridiculous, irrelevant formality?

Lesson two: The drug war accounts for almost all criminal wiretaps. Wiretaps may be down a bit in 2011, but over the long term they’ve still increased massively. Since 1997, even as communication has migrated from telephone networks to the internet on a mass scale, the annual number of wiretaps has more than doubled. And as this handy chart assembled by security researcher Chris Soghoian shows, our hopeless War on Drugs is driving almost all of it: for fully 85 percent of wiretaps last year, a drug offense was the most serious offense listed on the warrant application—compared with “only” 73 percent of wiretaps in 1997. Little surprise there: when you try to criminalize a transaction between a willing seller and a willing buyer, enforcement tends to require invasions of privacy. Oddly, law enforcement officials tend to gloss over these figures when asking legislators for greater surveillance authority. Perhaps citizens wouldn’t be as enthusiastic about approving these intrusive and expensive spying powers if they realized they were used almost exclusively to catch dope peddlers rather than murderers or kidnappers.

Speaking of dubious claims, lesson three: The encryption apocalypse is not nigh. As those of you who are both extremely nerdy and over 30 may recall, back in the 1990s we had something called the “Crypto Wars.” As far as the U.S. government was concerned, strong encryption technology was essentially a military weapon—not the sort of thing you wanted to allow in private hands, and certainly not something you could allow to be exported around the world. Law enforcement officials (and a few skittish academics) warned of looming anarchy unless the state cracked down hard on so-called “cypherpunks.” The FBI’s Advanced Telephony Unit issued a dire prediction in 1992 that within three years, they’d be unable to decipher 40 percent of the communications they intercepted.

Fortunately, they lost, and strong encryption in private hands has become the indispensable foundation of a thriving digital economy—and a vital shield for dissidents in repressive regimes. Frankly, it would probably have been worth the tradeoff even if the dire predictions had been right. But as computer scientist Matt Blaze observed back when the 2010 wiretap report was released, Ragnarok never quite arrives. The latest numbers show that investigators encountered encryption exactly 12 times in all those thousands of wiretaps. And how many times did that encryption prevent them from accessing the communication in question? Zero. Not once.

Now, to be sure, precisely because police seldom use wiretap orders for e-mail, that’s also a highly incomplete picture of the cases where investigations run up against encryption walls. But as the FBI once again issues panicked warnings that they’re “going dark” and demands that online companies be requried to compromise security by building surveillance backdoors into their services, it’s worth recalling that we’ve heard this particular wolf cry before. It would have been a disastrous mistake to heed it back then, and on the conspicuously scanty evidence being offered during the encore, it would be crazy to approach these renewed demands with anything less than a metric ton of salt.

Why Data Transparency?

At a recent Capitol Hill briefing on government transparency, I made an effort to describe the importance of getting data from the government reflecting its deliberations, management, and results.

I analogized to the World Wide Web. The structure that allows you to find and then view a blog post as a blog post is called hypertext markup language, or html. HTML is what made the Internet into the huge, rollicking information machine you see today. Think of the darkness we lived in before we had it.

Government information is not yet published in useable formats—as data—for the public to use as it sees fit. We need government information published as data, so we can connect it in new ways, the way the World Wide Web allowed connections among documents, images, and sounds.

And when you connect data together, you get power in a way that doesn’t happen with the web, with documents. You get this really huge power out of it.

Tim Berners-Lee was not thinking of wresting power from government when he said that, but the inventor of the World Web does a better job than I could of arguing for getting data and making it available for any use. We’ll look back on today with bemusement and surprise at the paucity of information we had about our government’s activities and expenditures.

Pages