It was, of course, a description that fit THINTHREAD perfectly—except for the collection and storage of terabytes of digital junk. THINTHREAD’s focus on metadata mining and link analysis was designed to help analysts pinpoint the truly important leads to follow while discarding irrelevant data. Hayden’s concept mirrored that of his successor, Keith Alexander, who also had a “collect it all” mentality.
In his memoir, Hayden spoke of the need to “engage industry” (p. 20) in the effort to help NSA conquer the challenge of sorting through the mind-numbing quantity of digital data, but even Hayden admitted that “When we went to them for things nobody had done yet, we found that at best they weren’t much better or faster than we were” (page 20).
That should’ve been Hayden’s clue that NSA would be better off pursuing full deployment of THINTHREAD, a proven capability. But Hayden chose to pursue his industry-centric approach instead, and he tolerated no opposition or second-guessing of the decision he’d made.
In April 2000, Hayden’s message to the NSA workforce made it clear that any NSA employees who went to Congress to suggest a better way for the NSA to do business would face his wrath. Even so, the THINTHREAD team pressed on, managing to get their system deployed to at least one NSA site in a test bed status, working against a real-world target. Meanwhile, Roark continued to push NSA to make the program fully operational, but Hayden refused, and just three weeks before Sept. 11, 2001, further development of THINTHREAD was terminated in favor of the still hypothetical TRAILBLAZER program.
DoD IG Investigation vs. Hayden’s memoir
As Loomis noted in his own account of the THINTHREAD-TRAILBLAZER saga, within days after the 9/11 attacks, NSA management ordered key components of THINTHREAD—the system Hayden had rejected—to be integrated (without the inclusion of 4th Amendment compliance software) into what would become known as the STELLAR WIND warrantless surveillance program. Terrified that the technology they’d originally developed to fight foreign threats was being turned on the American people, Loomis, Binney, and Wiebe retired from the NSA at the end of October 2001.
Over the next several months, they would attempt to get the Congressional Joint Inquiry to listen to their story, but to no avail. By September 2002, the trio of retired NSA employees, along with Roark, decided to file a Defense Department Inspector General (DoD IG) hotline complaint, in which they alleged waste, fraud, and abuse in the TRAILBLAZER program. Inside NSA, they still had an ally—a senior executive service manager named Tom Drake, who had become responsible for the remnants of THINTHREAD after the SARC team had resigned. Drake became the key source for the subsequent DoD IG investigation, which resulted in a scathing, classified report completed in December 2004.
The TRAILBLAZER-THINTHREAD controversy subsequently surfaced in the press, and I followed the reporting on it while working as a senior staffer for then-Representative Rush Holt (D-N.J.), a HPSCI member at the time. Once Holt was appointed to the National Commission on Research and Development in the Intelligence Community, I asked for and received copies of the published DoD IG reports dealing with the THINTHREAD and TRAILBLAZER programs.
The 2004 report remains the most damning IG report I’ve ever read, and after Holt announced his departure from Congress in 2014, I decided to continue my own investigation into this episode as an analyst at the Cato Institute. In March 2015, I filed a FOIA request seeking not only the original 2004 DoD IG report, but all other documents relevant to the investigation.
After being stonewalled by DoD and NSA for nearly two years, Cato retained the services of Loevy and Loevy of Chicago to prosecute a FOIA lawsuit to help get the documents I sought. In July 2017, the Pentagon released to me a still heavily redacted version of the 2004 DoD IG report. But there are fewer redactions in my copy than there were in the version provided to the Project on Government Oversight (POGO) in 2011, and it provides the clearest evidence yet that Hayden’s account of the THINTHREAD-TRAILBLAZER episode in his memoir is simply not to be believed.
On The IG Investigation Itself
On page 26 of his memoir, Hayden’s only mention of the IG investigation is a single sentence: “Thin Thread’s advocates filed an IG (inspector general) complaint against Trailblazer in 2002.”
Hayden makes no mention of the efforts he and his staff made to downplay THINTHREAD to the IG, or the climate of fear that Hayden and his subordinates created among those who worried TRAILBLAZER was a programmatic train wreck, and that THINTHREAD could, in fact, provide NSA with exactly the critical “finding the needle in the haystack” capability it needed in the digital age.
In its Executive Summary (page ii), the DoD IG report agreed THINTHREAD was the better solution and should be deployed:
And the DoD IG made it clear that NSA management—meaning Hayden—had deliberately excluded THINTHREAD as an alternative to TRAILBLAZER at a clear cost to taxpayers:
On Defying Congress
Hayden's fury at the SARC team keeping HPSCI staffer Roark in the loop about their progress was palpable, as he made clear on page 22 of his book: