Barack Obama pledged to preside over the “most transparent administration in history,” drawing an explicit contrast with the extreme secrecy of his predecessor. The Web site of the Department of Justice highlights that pledge, declaring its commitment to faithfully carry out a presidential directive encouraging such transparency, especially with regards to Freedom of Information Act requests, which are a vital tool for public accountability and informed democratic deliberation about government’s activities. Earlier this summer, I decided I’d put that commitment to what should have been an easy test.

When Congress passed the controversial FISA Amendments Act of 2008, granting the NSA broad power to conduct sweeping electronic surveillance of Americans’ international communications without individualized search warrants, it wisely required the Justice Department to issue semi-annual reports to Congress on the government’s implementation of the law, evaluating compliance with the various rules, guidelines, and procedures in place to reduce the risk of civil liberties abuses. While these reports are classified, redacted versions of several previous installments have been released to the public in response to Freedom of Information Act requests. The most recent is from May of 2010, which means that by now there are three or four further reports on the government’s use of its new spying powers which haven’t been seen by the public.

Since the FAA is set to expire at the end of this year, and Congress is rapidly steamrolling toward reauthorizing the law for another five years, it seems like now would be a good time to let the public see the latest versions of these reports—with any specific references to operational details removed, of course. That’s especially true given that we’ve recently learned that at least one ruling by the secretive FISA Court found some surveillance under the FAA had violated the Fourth Amendment. The latest reports, even in redacted form, might give us further insight into the scale and seriousness of this violation of Americans’ constitutional rights. If, on the other hand, we find no mention of this in the official reports, it would be powerful evidence that Congress is getting a whitewashed account, and that internal oversight may not provide adequate protection for our privacy and liberties. Again, the government has already released several previous installments of this report—though the ACLU ultimately had to file a lawsuit before they agreed to do so—so there should be no doubt now as to whether these are documents they’re obligated to release.

On June 26, therefore, I sent a FOIA request to the Justice Department asking for the release of the newer installments of this important report—specifically asking for expedited review, given the importance of informing the public about the use of the law before Congress renews it. On July 6, I got a response acknowledging that my request had been received and forwarded to the FOIA office of the DOJ’s National Security Division. Federal law requires agencies to reply to these requests within 20 business days. I was still waiting when, a few days ago, a bill extending FAA spying authority was scheduled for consideration before the House of Representatives this week. I did, however, have a brief phone conversation with the NSD’s FOIA officer confirming that she was evaluating my request, and that she understood clearly exactly which reports I was requesting.

This morning, September 10—more than two months after acknowledging receipt of my request for these three or four documents—I finally got a reply (my emphasis added), denying my request with the following unhelpful boilerplate:

The Office of Intelligence (OI) maintains operational files which consist of copies of all FISA applications, as well as requests for approval of various foreign intelligence and counterintelligence collection techniques such as physical searches.  We did not search these records in response to your request because the existence or nonexistence of such records on specific persons or organizations is properly classified under Executive Order 13526.  To confirm or deny the existence of such materials in each case would tend to reveal which persons or organizations are the subjects of such requests.  Accordingly, we can neither confirm nor deny the existence of records in these files responsive to your request pursuant to 5 U.S.C. §552(b) (1).

This is, in a word, ridiculous. The “existence” of the reports I asked for is required by federal law. To the extent they contain passing references to any specific persons or organizations under investigation, these can easily be redacted, and have been redacted for previous public releases of the same documents. No reasonable person could believe that this reply is applicable to my request. If it had been sent immediately, you could at least put it down to sloppiness or inattention, but remember, it took them two months to send out a denial based on the preposterous claim that it is classified information whether a report mandated by federal statute even exists.

I can appeal—and of course, I intend to—but since that’s likely to drag out the process for at least another month or two, the reports are likely to come too late to be relevant to the debate over FAA reauthorization. Try as I might, it’s almost impossible for me to see this as a good faith response to my request. Instead, it looks an awful lot like a stalling tactic calculated to drag out the process until it’s too late for the documents to be relevant to the debate over the FAA. I suppose this shouldn’t be terribly surprising: DOJ’s modus operandi, at least when it comes to anything controversial or potentially embarrassing to the government, seems to be to force FOIA requesters to waste time, energy, and money going to court even when it’s painfully obvious there’s no legitimate legal basis for sustaining a denial. That this is routine enough to be predictable, however, shouldn’t make it any more acceptable in a democracy.

And when there’s a bill that ends up on my desk as president, you, the public, will have five days to look online and find out what’s in it before I sign it, so that you know what your government’s doing.

When candidate Obama spoke that line on the campaign trail (starting around 1:00 in the video), it was met with a hail of applause. This same promise was featured on his campaign web site. He laid out a vision for a transparent, responsive government.

The interesting thing about this particular promise—what made it a rarity in campaigns—was that it was measurable. We could learn by watching whether President Obama would deliver on this promise.

So I did.

Over the last three-and-a-half years, I’ve taken note of when Congress has presented bills to the president, when they’ve been posted (accessibly) on Whitehouse.gov, and when the president has signed them.

This is my thirty-seventh post on the topic. Each character in that last sentence is a link to a previous post. Read through to see the full saga of Sunlight Before Signing implementation.

But enough chatter. How has the president done on a simple, straightforward campaign promise that would make the government more open and transparent?

The current tally is…(drumroll please)…

just under two-thirds compliance!

As the summary table below illustrates, the Obama White House almost completely disregarded the Sunlight Before Signing promise in the first year. In fact, it was the president’s first broken promise. Since then, the president has improved, but not by so much that people have come to rely on Sunlight Before Signing to “know what [their] government’s doing.”

Here’s part of why they don’t rely on it: We learned through study that the important bills tend notto get five days of public review before the president signs them. The ones that almost always do are the bills to rename post offices, change the borders of national parks, and such.

When the president promised Sunlight Before Signing, it was possible the strict adherence to his pledge from the first days of his presidency would have encouraged people to use the process as a tool for government oversight. I disagree with the point that bills sent to the president are faits accomplis. Sunlight Before Signing would have changed the “upstream” behavior of legislators who would not want to be caught out inserting the earmark or parochial amendment that takes down a bill.

Now the question stands: Will President Obama continue to implement Sunlight Before Signing in a second term? If so, will he apply the rule to all bills so that people can rely on having five days to review the legislation Congress sends him?

A legitimate alternative for President Obama is to swear off that campaign promise as improvidently made. Arguably, it was. Since then, we have learned through study (my study, in particular) that transparency is a set of practices making data reliably available and machine-discoverable and -readable.

If President Obama wants to do something for transparency, he could do something as simple as publish a federal government organization chart in machine-readable format so that legislation, oversight, and spending data could be linked to the unique identifiers for agencies, bureaus, programs, and projects, as well as authorizations, obligations, and outlays.

The Office of Management and Budget is becoming conspicuous for its foot-dragging in this area, and even its opposition to steps that would improve government transparency. If President Obama wants to be a transparency president, he could perfectly well shed Sunlight Before Signing and pledge in this campaign to make at least the organizational data OMB holds in the MAX database available to the public.

Needless to say, the Romney campaign can highlight President Obama’s inability to deliver on his soaring transparency promises by pledging to take the concrete steps noted above. I’d report just as doggedly on his compliance with such a promise.

While we wait, here are all the bills passed so far in the 112th Congress and their treatment under President Obama’s Sunlight Before Signing promise.

[Brackets indicate a link from Whitehouse.gov to Thomas legislative database]

‡ Link to final version of bill on impossible-to-find page.

It turns out that a New York Police Department program that assembled large databases on the ordinary activities of  innocent Muslims, infiltrated student groups, and monitored sermons wasn’t just controversial—it was useless. As the Associated Press reports, the head of the NYPD’s Intelligence Division recently confirmed in a deposition that the Department’s “Demographics Unit”—a delightful euphemism for a team dedicated to spying predicated wholly on ethnicity, language, and religion—turned up no useful leads and gave rise to no terrorism investigations in its six years of operation.

At the risk of being a broken record, this is a reminder of how misleading it can be to discuss these topics under the rubric of “balancing liberty and security.” If government surveillance performs as advertised and yields a substantial security benefit, there’s a debate to be had over how much government intrusion we’re prepared to countenance as the price of that security. But that security benefit has to be proven, not assumed. If it can’t be demonstrated—and a fortiori, if all available evidence demonstrates there is no benefit—then it just should not be a serious question in a decent society whether it’s acceptable for police to keep tabs on all Urdu-speakers of Pakistani origin on the premise (endorsed by this official) that “most” of them are people “of concern” to the government. Just to put that “most” in context: There are some 15 million Pakistani Urdu-speakers worldwide, and about 50,000 legal residents of Pakistani descent in the New York metro area. Treating them all, by default, as potential terrorists who need to be watched would be offensive and ugly even if the policy occasionally yielded a useful piece of information. But to squander scarce law enforcement resources targeting a minority population without any useful results over six years?  How can that be anything but obscene?