Back in September, I rated Congress on how well it is publishing information about its deliberations and decisions. “Needs Improvement” was the understated theme.

Now we’re looking at the government’s publication of data that reflects budgeting, appropriations, and spending. “Needs improvement” isn’t just understated in this area. It’s really, really understated.

On the budgeting, appropriations, and spending transparency report card I’m putting out today, B+ is the best grade—and it goes to just half of one subject area. There are 2.5 Cs, 3 Ds, and 4 incompletes. This area needs improvement.

What is transparency, anyway? In my briefing paper, “Publication Practices for Transparent Government,” I wrote about the publication practices that support transparency. They are: authority, availability, machine-discoverability, and machine-readability. That means putting good data out from a consistent source in sensible ways, and, especially, structuring the data so that computers can interpret it.

You know what the World Wide Web is? It’s a whole bunch of structured data. If you want the kind of breakthrough in transparency for government data that the Web was for communications, you want the data structured right.

Our draft structure for data in this area is in our “Conceptual Data Model of the U.S. Federal Government Budgetary Process.” (HTML version, Word version)

Structured data doesn’t really exist yet in the area of budgeting, appropriating, and spending. The one bright spot is the president’s annual budget submission, which includes some information in a workable structure, but there is much room for improvement even there.

Because I’m so nice, I’ve given a lot of “incompletes” where I could have—and some say should have—given Fs. Believe it or not, there is NO federal government “organization chart” that is published in a way computers can use. That’s one of the building blocks of computerized oversight, and its absence is easily rectified.

When we return to these issues in the summer or fall of next year, and review more formally how Congress and the administration have done on transparency, I expect these things to be fixed. (Fear the blog post!)

In the meantime, here’s a run-down of the grades and why they were given. A Hill briefing today might be available online at the page for the event. (It’s somewhat symbolic that the room we have on Capitol Hill is ill-equipped for live-streaming, but we’re going to try.)

I’ve alternated in this post between “I” and “we” because I’ve gotten so much help on this. People from OMB Watch, the National Priorities Project, and the Sunlight Foundation have helped a great deal with this project, to name a few—and omit many others! The grades, the commentary, the errors, the misstatements, and omissions are all mine. And there are going to be plenty of gaps in this work. That’s why this is a blog post and not a formal Cato publication.

Publication Practices for Transparent Government: Budgeting, Appropriations, and Spending

How well can the Internet access data about the federal government’s budgeting, appropriating, and spending? In consultation with transparency experts, the Cato Institute’s director of information policy studies, Jim Harper, rated how Congress and the administration publish key spending-cycle data in terms of authoritative sourcing, availability, machine-discoverability, and machine-readability.

These criteria envision a world where there is one authoritative source for each category of information. Unfortunately, what spending data there is appears in a lot of sources that have grown up haphazardly. There might even be some sources we don’t know about. Future grades will undoubtedly reflect improvements in what researchers, reporters, websites, and the public at large can see and use, aided by their computers.

Agencies: I

Federal agencies are the “agents” of Congress and the president. They carry out federal policy and spending decisions. Accordingly, one of the building blocks of data about spending is going to be a definitive list of the organizational units that do the spending.

Is there such a list? Yes! It’s Appendix C of OMB Circular A-11, “Listing of OMB Agency/Bureau and Treasury Codes.” But this list is a PDF document that is found on the Office of Management and Budget website.

Believe it or not, there is NO federal government “organization chart” that is published in a way amenable to computer processing!

There are distinct identifiers for agencies in both the Treasury Department and the Office of Management and Budget. Either of these could be published as the executive branch’s definitive list of its agencies. This fruit is hanging so low that a gopher could snack on it without leaving its hole, but nobody seems to have thought of publishing data about the basic units of the executive branch online in a machine-discoverable and machine-readable format.

A pathological excess of generosity spurs us to give this category an “incomplete” rather than a straight F. We expect improvement in publication of this data, pronto.

Bureaus: I

The sub-units of agencies are bureaus, and the same situation applies to data about the offices where the work of agencies get divided up. Bureaus have identifiers. It’s just that nobody publishes a list of bureaus, their parent agencies, and other key information for the Internet-connected public to use in coordinating its oversight.

Again, an “incomplete” in this area will quickly convert to an F if this gap in data publication is not soon rectified.

Programs: I

The work of the government is parceled out for actual execution in programs. Like information about their parental units, the agencies and bureaus, data that identifies and distinguishes programs is not comprehensively published.

There is some information about programs available in usable forms. The Catalog of Federal Domestic Assistance website (www.cfda.gov) has useful aggregation of some information on programs, but the canonical guide to government programs, along with the bureaus and agencies that run them does not exist.

This is a little bit heavier a lift than agencies and bureaus—the number of programs exceeds the number of bureaus by something like an order of magnitude (much as the number of bureaus exceeds the number of agencies). And it might be that some programs have more than one agency/bureau parent. But today’s powerful computers can keep track of these things—they can count pretty high! And the government should figure out all the programs it has, keep that list up to date, and publish it for public consumption.

Until it does, the program category gets an “incomplete” and the threat of a future F. (Or maybe a D thanks to the CFDA.)

Projects: D-

Projects are where the rubber hits the road. These are the organizational vehicles the government uses to enter into contracts and create other obligations that deliver on government services.

Some project information gets published—we finally have an item that is not incomplete—but the publication is so bad that we give this area a low grade indeed.

Information about projects can be found. You can search for projects by name on USASpending.gov, and descriptions of projects appear in USASpending/FAADS downloads. (“FAADS” is the Federal Assistance Award Data System), but there is no canonical list of projects that we could find. There should be, and there should have been for a long time now.

The generosity and patience we showed with respect to agencies, budgets, and programs has run out. There’s more than nothing here, but programs get a D-.

Budget Documents — Congress: D / White House: B+

The president’s annual budget submission and the congressional budget resolutions are the planning documents that the president and Congress use to map the direction of government spending each year. These documents are published authoritatively, and they are consistently available, which is good. They are kind-of machine-discoverable, but they are not terribly machine-readable.

The appendices to the president’s budget are published in XML format, which vastly reduces the time it takes to work with the data in them. That’s really good. But the congressional budget resolutions have no similar organization, and there is low correspondence between the budget resolutions that Congress puts out and the budget the president puts out. You would think that a person—or better yet, a computer—should be able to lay these documents side by side for comparison, but you can’t.

For its use of XML, the White House gets a B+. Congress gets a flat D.

Budget Authority—Congress: C- / Executive Branch: D

“Budget authority” is a term of art for what probably should be called “spending authority.” It’s the power to spend money, created when Congress and the president pass a law containing such authority.

Proposed budget authority is pretty darn opaque. The bills in Congress that contain proposed budget authority are consistently published online—that’s good—but they don’t highlight budget authority in machine readable ways. No computer can figure out how much budget authority is out there in pending legislation.

Existing budget authority is pretty well documented in the Treasury Department’s FAST book (Federal Account Symbols and Titles). This handy resource lists Treasury accounts and the statutes and laws that provide their budget authority. The FAST book is not terrible, but the only form we’ve found it in is PDF. PDF is terrible.

Congress can do a lot better, but because some of the publication basics are there, we give it a C-. The administration gets a D for publishing the obscure FAST book in PDF.

Ideally, there would be a nice, neat connection from budget authority right down to every outlay of funds, and back up again from every outlay to its budget authority. These connections, published online in useful ways, would allow public oversight to blossom.

Warrants, Apportionments, and Allocations: I

After Congress and the president create budget authority, that authority gets divvied up to different agencies, bureaus, programs and projects. How well documented are these processes? Not well.

An appropriation warrant is an assignment of funds by the Treasury to a treasury account to serve a particular budget authority. It’s the indication that there is money in an account for an agency to obligate and then spend.

Where is warrant data? We can’t find it. Given Treasury’s thoroughness, it probably exists, but it’s just not out there for public consumption. We’ve again generously given this area an “incomplete.”

An apportionment is an instruction from the Office of Management and Budget to an agency about how much it may spend from a treasury account in service of given budget authority in a given period of time.

We haven’t seen any data about this, and we’re less sure that there is some. There should be. And we should get to see it. Incomplete.

An allocation is a similar division of budget authority by an agency into programs or projects. We don’t see any data on this either. And we should. Incomplete.

Step up, Executive Branch, or we’ll convert these incompletes to very low grades, indeed…

Obligations: C+

Obligations are the commitments to spend money into which government agencies enter. Things like contracts to buy pens, hiring of people to write with those pens, and much, much more.

There are several different data sources that reveal obligations: FAADS/FAADS+ and CFDA, for example. But their numbers don’t match up, and—unless you’re going to have each agency uniformly publish its own data—obligations shouldn’t be published in different places. It’s hard to consider either one authoritative (even if the law says they both are). FAADS+/FPDS (via USASpending.gov), CFDA, and FPDS (the Federal Procurement Data System) are online and stable, but they are potentially incomplete because not all agencies may report to them. The use of proprietary DUNS numbers also weakens them in terms of availability.

Just sorting through all the acronyms can get you down. Ask data experts to get into the quality of each data source, and you’ll be boggled by the questions regarding which agencies’ obligations are reported at which source, whether given sources dumb down the data by excluding small dollar amounts or by aggregating data about smaller agencies. Some sources are more timely than others. Etc. etc. etc.

All these issues frustrate transparency. Data about obligations is not clean, complete and well documented. The ideal is to have one source of obligation data that combines the strengths of all the existing sources and that includes every agency, bureau, program, and project. With a decent amount of data out there, though, useful for experts, this category gets a C+.

Parties: D+

Of course, you want to know where the money is going. That is what we’re calling the “parties” category. (“Parties” sounds kinda fun, don’t it?)

Right now, reporting on parties is dominated by the DUNS number. That’s the Data Universal Numbering System, which provides a unique identifier for each business entity. It was developed by Dun & Bradstreet in the 1960s. It’s very nice to have a distinct identifier for every entity doing business with the government, but it is not very nice to have the numbering system be a proprietary one.

Parties would grade well in terms of machine-readability, which is one of the most important measures of but because it scores so low on availability, its machine-readability is kind of moot. Until the government moves to an open identifier system for recipients of funds, it will get weak grades on publication of this essential data.

Outlays: C-

For a lot of folks, the big kahuna is knowing where the money goes: outlays. An outlay—literally, the laying out of funds—satisfies an obligation. It’s the movement of money from the U.S. Treasury to the outside world.

Outlay numbers are fairly well reported after the fact and in the aggregate. All you have to do is look at the appendices to the president’s budget to see how much money has been spent in the past.

But outlay data can be much, much more detailed and timely than that. Each outlay goes to a particular party. Each outlay is done on a particular project or program at the behest of a particular bureau and agency. And each outlay occurs because of a particular budget authority. Right now these details about outlays are nowhere to be found.

Now, there are plenty of people inside the government who are very familiar with the movement of taxpayer money in the government. They will be inclined to say, “it’s more complicated than that,” and it is! But it’s going to have to get quite a bit less complicated before these processes can be called transparent.

The time do de-complicate outlays is now. It’s another feat of generosity to give this area a C-. That’s simply because there is an authoritative source for aggregate past outlay data. As the grades other areas come up, outlay data that stays the same could go down. Waaaayyy down.