Just How Closely Can The NSA Really Watch You?

Brian Proffitt | ReadWrite | June 20, 2013

Leaks that suggest the NSA is vacuuming up personal information from the phone records and online-service data of U.S. citizens have some people concerned about the prospect of an Orwellian surveillance state that can track our every move.

But hold on a second. More recent revelations suggest that intelligence agencies are using Hadoop and other well-established Big Data business tools to run their analyses. And that suggests there are actually technical limits on how much the NSA can actually track and when.

So let's run through a brief technical thought experiment. Just how much can the government learn about you using Hadoop?

The Limits Of Hadoop

Based on sources close to the NSA, the Wall Street Journal has reported that the agency is very likely making use of the data gathering and analysis platform Hadoop as well as non-relational database technology — those NoSQL databases you've heard tell of.