Tag: airline security

Are You Substituting Worst-Case Thinking for Reason?

Bruce Schneier has a typically good essay on the use of “worst-cases” as a substitute for real analysis. I noticed conspicuous use of “worst-case” in early reporting on the oil spill in the Gulf. It conveniently gins up attention for media outlets keen on getting audience.

There’s a certain blindness that comes from worst-case thinking. An extension of the precautionary principle, it involves imagining the worst possible outcome and then acting as if it were a certainty. It substitutes imagination for thinking, speculation for risk analysis and fear for reason. It fosters powerlessness and vulnerability and magnifies social paralysis. And it makes us more vulnerable to the effects of terrorism.

Worst-case thinking—the failure to manage risk through analysis of costs and benefits—is what makes airline security such an expensive nightmare, for example. Schneier concludes:

When someone is proposing a change, the onus should be on them to justify it over the status quo. But worst case thinking is a way of looking at the world that exaggerates the rare and unusual and gives the rare much more credence than it deserves. It isn’t really a principle; it’s a cheap trick to justify what you already believe. It lets lazy or biased people make what seem to be cogent arguments without understanding the whole issue.

It’s not too long for you to read the whole thing.

Making Sense of New TSA Procedures

Since they were announced recently, I’ve been working to make sense of new security procedures that TSA is applying to flights coming into the U.S.

“These new measures utilize real-time, threat-based intelligence along with multiple, random layers of security, both seen and unseen, to more effectively mitigate evolving terrorist threats,” says Secretary Napolitano.

That reveals essentially nothing of what they are, of course. Indeed, “For security reasons, the specific details of the directives are not public.”

But we in the public aren’t so many potted plants. We need to know what they are, both because our freedoms are at stake and because our tax money will be spent on these measures.

Let’s start at the beginning, with identity-based screening and watch-listing in general. A recent report in the New York Times sums it up nicely:

The watch list is actually a succession of lists, beginning with the Terrorist Identities Datamart Environment, or TIDE, a centralized database of potential suspects.  … [A]bout 10,000 names come in daily through intelligence reports, but … a large percentage are dismissed because they are based on “some combination of circular reporting, poison pens, mistaken identities, lies and so forth.”

Analysts at the counterterrorism center then work with the Terrorist Screening Center of the F.B.I. to add names to what is called the consolidated watch list, which may have any number of consequences for those on it, like questioning by the police during a traffic stop or additional screening crossing the border. That list, in turn, has various subsets, including the no-fly list and the selectee list, which requires passengers to undergo extra screening.

The consolidated list has the names of more than 400,000 people, about 97 percent of them foreigners, while the no-fly and selectee lists have about 6,000 and 20,000, respectively.

After the December 25, 2009 attempted bombing of a Northwest Airlines flight from Amsterdam into Detroit, TSA quickly established, then quickly lifted, an oppressive set of rules for travelers, including bans on blankets and on moving about the cabin during the latter stages of flights. In the day or two after a new attempt, security excesses of this kind are forgivable.

But TSA also established identity-based security rules of similar provenance and greater persistence, subjecting people from fourteen countries, mostly Muslim-dominated, to special security screening. This was ham-handed reaction, increasing security against the unlikely follow-on attacker by a tiny margin while driving wedges between the U.S. and people well positioned to help our security efforts.

Former DHS official Stewart Baker recently discussed the change to this policy on the Volokh Conspiracy blog:

The 14-country approach wasn’t a long-term solution.  So some time in January or February, with little fanfare, TSA seems to have begun doing something much more significant.  It borrowed a page from the Customs and Border Protection playbook, looking at all passengers on a flight, running intelligence checks on all of them, and then telling the airlines to give extra screening to the ones that looked risky.

Mark Ambinder lauded the new policy on the Atlantic blog, describing it thusly:

The new policy, for the first time, makes use of actual, vetted intelligence. In addition to the existing names on the “No Fly” and “Selectee” lists, the government will now provide unclassified descriptive information to domestic and international airlines and to foreign governments on a near-real time basis.

Likely, the change is, or is very much like, applying a Customs and Border Patrol program called ATS-P (Automated Targeting System - Passenger) to air travel screening.

“[ATS-P] compares Passenger Name Record (PNR) and information in [various] databases against lookouts and patterns of suspicious activity identified by analysts based upon past investigations and intelligence,” says this Congressional Research Service report.

“It was through application of the ATS-P that CBP officers at the National Targeting Center selected Umar Farouk Abdulmutallab, who attempted to detonate an explosive device on board Northwest Flight 253 on December 25, 2009, for further questioning upon his arrival at the Detroit Metropolitan Wayne County Airport.”

Is using ATS-P or something like it an improvement in the way airline security is being done? It probably is.

A watch-list works by comparing the names of travelers to the names of people that intelligence has deemed concerning. To simplify, the logic looks like something like this:

If first name=”Cat” (or variants) and last name=”Stevens”, then *flag!*

Using intelligence directly just broadens the identifiers you use, so the comparison (again simplified) might look something like this:

If biography contains “traveled in Yemen” or “Nigerian student” or “consorted with extremists”, then *flag!*

The ability to flag a potential terrorist with identifiers beyond name is a potential improvement. Such a screening system would be more flexible than one that used purely name-based matching. But using more identifiers isn’t automatically better.

The goal—unchanged—is to minimize both false positives and false negatives—that is, people flagged as potential terrorists who are not terrorists, and people not flagged as terrorists who are terrorists. A certain number of false positives are acceptable if that avoids false negatives, but a huge number of false positives will just waste resources relative to the margin of security the screening system creates. Given the overall paucity of terrorists—which is otherwise a good thing—it’s very easy to waste resources on screening.

I used the name “Cat Stevens” above because it’s one of several well known examples of logic that caused false positives. Utterly simplistic identifiers like “traveled in Yemen” will also increase false positives dramatically. More subtle combinations of identifiers and logic can do better. The questions are how far they increase false positives, and whether the logic is built on enough information to produce true positives.

So far as we know, ATS-P has never flagged a terrorist before it flagged the underwear bomber. DHS officials tried once to spin up a case in which ATS-P flagged someone who was involved in an Iraq car-bombing after being excluded from the U.S. However, I believe, as I suggested two years ago, that ATS-P flagged him as a likely visa overstayer and not as a terror threat. He may not have been a terror threat when flagged, as some reports have it that he descended into terrorism after being excluded from the U.S. This makes the incident at best an example of luck rather than skill. That I know of, nobody with knowledge of the incident has ever disputed my theory, which I think they would have done if they could.

The fact that ATS-P flagged one terrorist is poor evidence that it will “work” going forward. The program “working” in this case means that it finds true terrorists without flagging an unacceptable/overly costly number of non-terrorists.

Of course, different people are willing to accept different levels of cost to exclude terrorists from airplanes. I think I have come up with a good way to measure the benefits of screening systems like this so that costs and benefits can be compared, and the conversation can be focused.

Assume a motivated attacker that would eventually succeed. By approximating the amount of damage the attack might do and how long it would take to defeat the security measure, one can roughly estimate its value.

Say, for example, that a particular attack might cause one million dollars in damage. Delaying it for a year is worth $50,000 at a 5% interest rate. Delaying for a month an attack that would cause $10 billion in damage is worth about $42 million.

(I think it is fair to assume that any major attack will happen only once, as it will produce responses that prevent it happening twice. The devastating “commandeering” attack on air travel and infrastructure is instructive. The 9/11 attacks permanently changed the posture of air passengers toward hijackers, and subsequent hardening of cockpit doors has brought the chance of another commandeering attack very close to nil.)

A significant weakness of identity-based screening (which “intelligence-based” screening—if there’s a difference—shares) is that it is testable. A person may learn if he or she is flagged for extra scrutiny simply by traveling a few times. A person who passes through airport security six times in a two-month period and does not receive extra scrutiny can be confident enough on the seventh trip that he or she will not be specially screened. If a person does receive special scrutiny on test runs, that’s notice of being in a suspect category, so someone else should carry out a planned attack.

(“We’ll make traveling often a ground for suspicion!” might go the answer. “False positives,” my rejoinder.)

Assuming that it takes two months more than it otherwise would to recruit and clear a clean-skin terrorist, as Al Qaeda and Al Qaeda franchises have done, the dollar value of screening is $125 million. That is the amount saved (at a 5% interest rate) by delaying for one month an attack costing $15 billion (a RAND corporation estimate of the total cost of a downed airliner, public reactions included).

Let’s say that the agility of having non-name identifiers does improve screening and causes it to take three months rather than two to find a candidate who can pass through the screen. Ignoring the costs of additional false positives (though they could be very high), the value of screening rises to $187.5 million.

(There is plenty of room to push and pull on all these assumptions. I welcome comments on both the assumptions and the logic of using the time-value of delayed attacks to quantify the benefits of security programs.)

A January 2009 study entitled, “Just How Much Does That Cost, Anyway? An Analysis of the Financial Costs and Benefits of the ‘No-Fly’ List,” put the amount expended on “no-fly” listing up to that time at between $300 million and $966 million, with a medium estimate of $536 million. The study estimated yearly costs at between $51 and $161 million, with a medium estimate of $89 million.

The new screening procedures, whose contours are largely speculative, may improve air security by some margin. Their additional costs are probably unknown to anyone yet as false positive rates have yet to be determined, and the system has yet to be calibrated. Under the generous assumption that this change makes it 50% harder to defeat the screening system, the value of screening rises, mitigating the ongoing loss that identity-based screening appears to bring to our overall welfare.

Hey, if you’ve read this far, you’ll probably go one or two more paragraphs…

It’s worth noting how the practice of “security by obscurity” degrades the capacity of outside critics to contribute to the improvement of homeland security programs. Keeping the contours of this system secret requires people like me to guess at what it is and how it works, so my assessment of its strengths and weaknesses is necessarily degraded. As usual, Bruce Schneier has smart things to say on security by obscurity, building on security principles generated over more than 125 years in the field of cryptography.

DHS could tell the public a great deal more about what it is doing. There is no good reason for the security bureaucracy to insist on going mano a mano against terrorism, turning away the many resources of the broader society. The margin of information the United States’ enemies might access would be more than made up for by the strength our security programs would gain from independent criticism and testing.

Making Airline Travel as Unpleasant as Possible

The Transportation Safety Administration long has made air travel as unpleasant as possible without obvious regard to the impact on safety.  Thankfully, the TSA recently dropped the inane procedure of asking to see your boarding pass as you passed through the checkpoint – a few feet away from where you entered the security line, at which point you had shown both your boarding pass and ID. 

However, there are proposals afoot in Congress to set new carry-on luggage restrictions, to be enforced by the TSA, even though they would do nothing to enhance security.  An inch either way on the heighth or width of a bag wouldn’t help any terrorists intent on taking over an airplane.  But the proposed restrictions would inconvenience travelers and allow the airlines to fob off on government what should be their own responsibility for setting luggage standards. 

TSA also has restarted ad hoc inspections of boarding passengers.  At least flights as well as passengers are targeted randomly.  After 9/11 the TSA conducted secondary inspections for every flight.  The process suggested that the initial inspections were unreliable, delayed passengers, and led experienced flyers to game the process.  It was critical to try to hit the front of the line while the inspectors were busy bothering someone else.  There was no full-proof system, but I learned that being first or second in line was particularly dangerous.

Finally TSA dropped the practice.  And, as far as I am aware, no planes were hijacked or terrorist acts committed as a result.  But TSA recently restarted the inspections, though on a random basis.

I had to remember my old lessons last week, when I ran into the routine on my return home from a trip during which I addressed students about liberty.  Luckily I was able to get on board, rather than get stuck as TSA personnel pawed through bags already screened at the security check point.

There’s no fool-proof way to ensure security for air travel.  Unfortunately, it’s a lot easier to inconvenience passengers while only looking like one is ensuring airline security.

TSA Search Overturned

A federal judge just threw out three fake passports discovered by a Transportation Security Agency (TSA) screener, holding that the search exceeded the TSA’s aviation security mission. (H/T Bruce Schneier)

This is long overdue; the TSA has moved beyond its original mandate and is now conducting searches for “contraband.” The search for anything that seems suspicious can quickly turn into an inquisition at the security checkpoint. Campaign for Liberty staffer Steven Bierfeldt experienced this at the St. Louis airport, and is now suing to prevent future searches beyond what is necessary for aviation security.

The invasive searches don’t add much to airline security anyway. Just as GAO investigators consistently defeat security at federal buildings, TSA screeners often fail to find fake explosives on security test teams.

As Bruce Schneier points out in his excellent book, Beyond Fear: Thinking Sensibly About Security in an Uncertain World, the two effective changes in airline security since September 11, 2001 have been (1) hardening of cockpit doors; and (2) airline passengers will resist because they know that their hijackers are playing for keeps.

Schneier spoke at Cato’s two-day conference on counterterrorism in January. Video at the link.

It Is a Checkpoint, After All

The Philadelphia Inquirer asks why the TSA didn’t catch Bonnie Sweeten absconding to Orlando at the airport after faking her own and her daughter’s abduction.

The TSA and FBI are right: it’s not airport security’s job to look for people like Bonnie Sweeten. But they will quickly agree to make it part of their mission when newspapers and Members of Congress start to say they should. This is how a nominal airline security program transmogrifies into a general law enforcement checkpoint, and the noose tightens on your right to travel.