Last week, I wrote about how crude algorithms in the name-matching “No Fly” system produce an outsize number of false positives as a matter of deliberate policy. We are willing to tolerate additional delays so that we can stop terrorists from flying. Yesterday, the DOJ’s Office of the Inspector General issued a report that seriously calls into question the bargain that we have struck with regard to the “No Fly” system. The report explains that the FBI (the agency amassing the list that is then matched to travelers’ names) has incorrectly kept 24,000 names on the terrorist watch list on the basis of outdated and irrelevant information, while “missing people with genuine ties to terrorism who should have been on the list.” According to the report, these mistakes not only posed a risk to national security due to the failure to flag actual terrorist suspects, but also created unnecessary delays and detentions for innocent travelers. A fact of great concern: the Inspector General sampled 216 FBI terrorism investigations and found that in 15% of them, a total of 35 subjects were not referred to the list even though they should have been.
During a talk that I gave last week for Princeton University’s Center on Information Technology Policy, Ed Felten (who served on TSA’s Secure Flight Study Group where he studied the No-Fly mechanism) explained that there are two aspects to the no-fly list, one that puts names on the list and the other that checks airline reservations against the list. The two parts operate separately from each other. The FBI heads up the first part, putting names on the list through a secret process that seemingly requires that people on the list be a sufficiently serious threat to aviation security. The other part is the one that I wrote about last week: a data-matching system that checks travelers’ names against the list. Because the matching algorithm requires only an approximate match (because flight reservations so often have misspelled names), we have many false positives so that we can sweep within the system the right match, i.e., the terrorist suspect, along with many innocent others.
So here is the rub: we are willing to live with so many false positives because we trust those amassing the list to ensure that it is accurate and complete. In other words, it worth all of those false positives if indeed they serve the greater good. Yes, we will endure the delay and perhaps inability to fly if indeed our names are akin to someone’s who is correctly suspected to be a terrorist. But preventing innocent individuals from flying, or subjecting them to questioning, based on matches with other innocent people’s names while failing to do enough homework so that you let real terrorist subjects board airplanes with no hassle? Really? This report suggests reconsidering having a “No Fly” system in its current form at all.
Thanks to Wikimedia Commons for the picture