Published Sunday, November 3, 2013 at 1:00 am / Updated at 12:08 pm
David Ignatius: Too much chatter can drown out vital intel

Several years ago, the Intelligence Advanced Research Projects Activity, an in-house think tank for the intelligence community, launched what is known as the “Good Judgment Project.” The idea is to test through forecasting competitions the factors that lead analysts to make good predictions.

One of the most interesting findings, according to a participant in the project, is that forecasting accuracy doesn’t necessarily improve when analysts have access to highly classified signals — intelligence of the sort the National Security Agency has been collecting secretly from the phones and messages of world leaders and, it sometimes seems, nearly everyone else. In fact, the top forecasters, drawn from universities and elsewhere, performed about 30 percent better than the average for intelligence community analysts who could read intercepts and other secret data.

Good predictions don’t necessarily correlate with access to secret data, in other words. Indeed, according to Philip Tetlock, a University of Pennsylvania professor who heads the project, too much information can sometimes overwhelm analysts and decrease their forecasting accuracy.

This problem of separating the essential “signal” from the chatter of “noise” has been well known to scholars for decades. But it seems to have been forgotten by modern-day intelligence agencies in their push to collect.

The NSA obviously operates on the theory that more data is better. And it has been ingenious at expanding the information available for analysts. If there’s a thrice-encrypted channel such as “The Onion Router,” NSA will crack it. If there’s a privacy scheme such as “secure sockets layer,” NSA will decrypt it. If there’s a world leader with an accessible cellphone, the NSA will tap it. But this mad dash for signals lacks the essential quality of sound judgment.

Reading the daily revelations of NSA electronic surveillance, some people have likened the situation to George Orwell’s all-seeing “Big Brother” in his novel “1984.” That image doesn’t match my own impression of the egghead military leaders of the NSA. Rather, I think of the “sorcerer’s apprentice” in Walt Disney’s movie “Fantasia,” who created a disaster by using tools and spells whose consequences he didn’t understand.

The NSA documents that have surfaced reveal an exuberant, almost adolescent quality among the tech wizards as they blow through privacy barriers. They give their top-secret projects colorful code names such as “Boundless Informant” or “Egotistical Giraffe.” They create compartments with mottos that sound like playground boasts: “Nothing but net” or “The mission never sleeps.”

Hannah Arendt spoke famously of the banality of evil. This group makes one realize that childishness can be a characteristic, too. Like many hackers, NSA operatives seem to have done things sometimes for the thrill of it, because they could.

NSA leaders defend themselves by arguing that they need the largest possible sample of “metadata,” registering calls and messages, so they can later sift this haystack looking for the dangerous needles.

The problem is that the NSA seems unwilling to recognize any limits on how big the haystack can be. The Foreign Intelligence Surveillance Court flatly warned in 2011 that some of the agency’s activities were illegally broad; the NSA supposedly changed its domestic procedures. But a report in Thursday’s Washington Post suggests that the agency used its foreign intelligence authority to collect similar information overseas. This should worry even people who support the NSA’s broad intelligence and counterterrorism missions.

The hemorrhage of secrets has come from leaks by former NSA contractor Edward Snowden. It’s hard to see him, even now, as any kind of hero: The way he chose to reveal programs that were legally authorized (albeit in some cases unwisely) has created severe problems for the United States and will cost tens of billions of dollars for U.S. companies that cooperated with court orders and NSA requests.

But it’s a fact that we are living now in a post-Snowden world and that U.S. policies for intelligence collection will need to be different.

It’s hard to imagine global agreement on a framework for spying, which by definition involves breaking other countries’ laws. But America and the many, many other countries that conduct surveillance do need new rules of the road. Conventions against torture, chemical weapons and abuse of prisoners don’t prevent wars. But they do limit extreme activities of combatants. Something similar is needed here.

The worst outcome for the global economy, alas, now seems most likely — which is that nations will try to ring-fence their data within national borders. That anti-globalization move won’t stop the spies, but it will slow commerce and innovation, and make digital life harder for everyone.

Contact the writer: davidignatius@washpost.com

Read more related stories
Crews working to subdue brush fire that may spread to Fontenelle Forest
Financial picture improving for city-owned Mid-America Center
No injuries after fire at midtown's old Mercer Mansion
29-year-old Omahan arrested for 22nd time in Lincoln
Police: Slaying of woman in Ralston apartment likely over drugs
Explosion near 29th, Woolworth damages vehicles
Omaha police arrest man, 19, accused in March shooting
Earth gets its day in the sun at Elmwood Park
Beau McCoy strikes Obama doll in TV ad; Democrats are not happy
17 senators in Nebraska Legislature hit their (term) limits
It's a pursuit of pastel at Spring Lake Park's Easter egg hunt
Keystone XL pipeline backers blast 'political expediency' as foes hail ruling to delay decision
Nebraska senators to study tax issues over break
Portion of Saddle Creek Road closed after water main break
Teenager arrested after woman's purse is snatched outside Omaha store
Police identify 21-year-old shot in ankle near 30th, W Streets
Cult murderer's death row appeal denied, but execution in limbo
Interstate construction to cause lane shifts, closings in Omaha area
Kelly: A California university president returns to her Nebraska roots on Ivy Day
Omahan charged in fatal shooting in Benson neighborhood
Friday's attendance dips at Millard West after bathroom threat
High school slam poets don't just recite verses, 'they leave their hearts beating on the stage'
Crack ring's leaders join others in prison as a result of Operation Purple Haze
Haze in area comes from Kansas, Oklahoma
Man taken into custody in domestic dispute
< >
COLUMNISTS »
Dickson’s Week in Review, April 13-19
On Twitter some guy tweeted that the spring game isn’t taken as seriously as a regular-season contest. What was your first clue? When the head coach entered waving a cat aloft?
Kelly: A California university president returns to her Nebraska roots on Ivy Day
The main speaker at today's Ivy Day celebration at the University of Nebraska-Lincoln is a college president who grew up roping calves and earned her Ph.D. at the prestigious Oxford University in England.
Breaking Brad: Stuck in a claw machine? You get no Easter candy
I know of one kid in Lincoln who will be receiving a lump of coal from the Easter Bunny, just as soon as he's extricated from that bowling alley claw machine.
Breaking Brad: Mountain lion season's over, but the bunny's fair game!
Thursday was the last day of a Nebraska Legislature session. Before leaving town, legislators passed a bill to hold a lottery to hunt the Easter Bunny.
Breaking Brad: At least my kid never got stuck inside a claw machine
We need a new rule in Lincoln. If your kid is discovered inside the claw machine at a bowling alley, you are forever barred from being nominated for "Mother of the Year."
Deadline Deal thumbnail
The Jaipur in Rockbrook Village
Half Off Fine Indian Cuisine & Drinks! $15 for Dinner, or $7 for Lunch
Buy Now
PHOTO GALLERIES »
< >
SPOTLIGHT »
Omaha World-Herald Contests
Enter for a chance to win great prizes.
OWH Store: Buy photos, books and articles
Buy photos, books and articles
Travel Snaps Photo
Going on Vacation? Take the Omaha World-Herald with you and you could the next Travel Snaps winner.
Click here to donate to Goodfellows
The 2011 Goodfellows fund drive provided holiday meals to nearly 5,000 families and their children, and raised more than $500,000 to help families in crisis year round.
WORLD-HERALD ALERTS »
Want to get World-Herald stories sent directly to your home or work computer? Sign up for Omaha.com's News Alerts and you will receive e-mails with the day's top stories.
Can't find what you need? Click here for site map »