Tomorrow’s editions of Time and Newsweek put the NSA’s phone database on their covers. Time’s story is emminently skippable, if you’ve been following the story at all. Newsweek does a much smarter job, offering a neat history of the NSA, and providing a needed antidote to the myth of the agency’s omnipotence.
But increasingly, there has been talk of the agency’s “going deaf.” The NSA had its best luck monitoring Soviet lines of communicationfor example, a microwave transmission from Moscow to a missile base in Siberia. But the new enemy is more shadowy and elusive. In 2002, General Hayden told NEWSWEEK, “We’ve gone from chasing the telecommunications structure of a slow-moving, technologically inferior, resource-poor nation-stateand we could do that pretty wellto chasing a communications structure in which an Al Qaeda member can go into a storefront in Istanbul and buy for $100 a communications device that is absolutely cutting edge, and for which he has had to make no investment for development.“
According to most accounts, the NSA remains behind the telecommunications curve. A December 2002 report by the Senate intelligence committee noted that only a “tiny fraction” of the NSA’s 650 million daily intercepts worldwide “are actually ever reviewed by humans, and much of what is collected gets lost in the deluge of data.” Hayden told NEWSEEK that year that the NSA had been slow to catch up to new technology, and that he was obsessed with turning the enemy’s “beeps and squeaks into something intelligible.“
One of Hayden’s most ambitious initiatives was called Trailblazer. It was a program aimed at helping the NSA make sense of its many databasesto put them to use. By more efficiently locating and retrieving messages, Trailblazer could help the NSA “data-mine,” to find patterns in the huge volume of electronic traffic that might help lead sleuths to a terror suspect. Instead, the program has produced nearly a billion dollars’ worth of junk hardware and software. “It’s a complete and abject failure,” says Robert D. Steele, a CIA veteran who is familiar with the program. Adds Ed Giorgio, who was the chief code breaker for the NSA for 30 years: “Everybody’s eyes rolled when you mentioned Trailblazer.“
What went wrong? The NSA apparently tried a clunky top-down approach, trying to satisfy too many requirements with one grand solution, rather than taking a more Silicon Valley-like tack of letting small entrepreneurs compete for ideas. John Arquilla of the Naval Postgraduate School at Monterey, Calif., a renowned “network” intelligence expert, says: “The real problem Big Brother is having is he’s not making enough use of the Little Brothers“the corporations that have become expert at manipulating databases for commercial use.
“Data mining” has been a boon to credit-card companies that can match customers and products. It has also helped the Feds track drug dealers who constantly buy and throw away cell phones (the technology can monitor frequent phone-number changes). Identifying and tracking terrorists may be a taller order. For one thing, terrorists have learned not to even use phones. A computer disk or message between, say, Osama bin Laden and Iraqi insurgent leader Abu Mussab al-Zarqawi is hand-delivered. Some terrorists have learned to leave messages hidden in Web sites. Others are given passwords to go on the Web sites and find the messages. Since that process involves no electronic communicationno e-mail or phone callthe NSA is kept in the dark.
Meanwhile, Newsweek’s technologist, Steven Levy, takes a page out of the Defense Tech playbook and chats with our pal Valdis Krebs. The Times gets proof for what Bobby Ray Inman told us on Monday: that Dick Cheney is the driving force behind the CIA program. The Washington Post catches DNI John Negroponte in an eavesdropping fib. The AP wraps a rather-hysterical headline (“Spy Agency Watching Americans From Space”) around a fairly sober look at the National Geospatial-Intelligence Agency. And News.com has a handy FAQ on the call record brouhaha. But my favorite NSA-related article from the weekend comes from the Chicago Trib’s technology ace, John Van, who talks to researchers about just how helpful all this link analysis and data mining might be.
The likelihood of success, Northwestern University’s Kris Hammond said, is higher if agents have specific questions, such as hypothetically what mobile phones in Washington, D.C., made calls to Tehran during a given period, and whether calls were made from those phones to San Francisco during another period.
But if officials don’t know what they’re looking for, they can’t expect a data mining program to connect all the dots.
“If you approach the data without specific questions and just look for patterns, you can find hundreds of millions of patterns,” Hammond said.
Despite advances in artificial intelligence, computers aren’t like human detectives who can make inferences and shift assumptions on the fly, said Yali Amit, a University of Chicago professor of statistics and computer science.
Government agents may not understand this, he said.
“They have records from millions of innocent people and perhaps a few thousand terrorists who might make phone calls,” said Amit. “The size of the data set of interest–the terrorists–is too small. You get reliability rates that make the whole endeavor pretty ridiculous.“
The White House hasn’t confirmed the NSA program, but in December, an official of DARPA, a Defense Department agency that funds advanced research, published a paper in an academic journal that suggests an ambitious role for link mining.
“Metaphorically, link mining offers the potential not only for connecting the dots, but for determining which dots to connect, a far more difficult task,” wrote Ted Senator, who stipulated he was expressing his own views, not those of DARPA or the government.
UPDATE 11:28 PM: Defense Tech pal Kim Zetter has a dynamite interview with intelligence historian Matthew Aid:
I’ll tell you where this story probably will go next. Notice the USA Today article doesn’t mention whether the Internet service providers or cellphone providers or companies operating transatlantic cables like Global Crossing cooperated with the NSA. That’s the next round of revelations. The real vulnerabilities for the NSA are the companies. Sooner or later one of these companies, fearing the inevitable lawsuit from the ACLU, is going to admit what it did, and the whole thing is going to come tumbling down…
The newest system being added to the NSA infrastructure, by the way, is called Project Trailblazer, which was initiated in 2002 and which was supposed to go online about now but is fantastically over budget and way behind schedule. Trailblazer is designed to copy the new forms of telecommunications — fiber optic cable traffic, cellphone communication, BlackBerry and Internet e-mail traffic.
{ 9 comments… read them below or add one }
Based on these two links, I’d say that the latest NSA program is quite legal, well at least if they buy them, like everybody else.
http://www.suntimes.com/output/news/cst-nws-privacy05.html
http://www.democrats.com/spy-on-them
This issue is moot.
This quote says it all
“Nobody whos not involved in terrorism should be at risk. Nobody whos making normal phone calls should be at risk. But the idea that were going to say to the United States government, for libertarian reasons, ‘Wed rather lose a city than have you gather data,’ I think is totally out of touch with the danger of the modern world.” – Newt Gingrich (14 May 2006)
You guys will be the first ones after the next strike on US soil running around screaming why we didn’t we see it coming, what failed, who’s to blame, why Bush didn’t put our full tools to detect the terrorist, lets have a commision to decide what technologies and processes could be used to stop the next strike. blah blah blah
When I was on the university, I played the Rappelz game under my classmates introduced. At first, I heard Rappelz from my classmate; I did not feel the game is fun. Until one day, I went on the line, because I had nothing to do, so I think I can look at the Rappelz game. When I looked the screen, I was attracted. Then I applied an account and I spent a little to buy rupees. And I began my Rappelz trip.
earn the 2moons dil and together play with friends, playing now I also feeling some tired, I do not know what things I persist in? Perhaps this 2moons game is no longer the game for me
they often like to ask something about the players looks like, family, and in the game can earn how much 2moons gold, I do not know these people are playing the game or game play them.
Look at the cabal alz; I think of many things, I very would like to return to the original place to say that in fact, I really love you very much.
in order to play this game I spend money to buy the flyff penya, the BB again left me at the same time, same situation when I hard to get it, my angry can not use the words describe, if I was not operated for a full and the lack of time, then I do not have any words to say, but last time also like this.
Other players can, I think I can too, many players said that BB can let us get more flyff gold, if you do not have enough flyff gold in the game, and then you can not do many things, in the reality world also like this. But I more like through my effort to get the success, that success can let me memorable forever.
If you think that play game was important that have you thought about your parents? I want to say that when you played game that you do not forget your parents and friends. When you used your parents give your money to buy the Archlord gold, did you thought that money was your parents