The National Security Agency secretly broke into the network connections between datacenters belonging to Yahoo, Google and other companies to drink data direct from the firehose, as well as collect metadata about the Internet use of U.S. residents, The Washington Post reported.
According to a document market Top Secret and dated Jan. 9, 2013, said the newspaper, the NSA’s acquisitions directorate was able to send millions of records per day from the internal networks connecting datacenters of major Internet companies to data warehouses in the agency’s headquarters in Fort Meade, which also houses one of the agency’s main data centers.
During the 30 days before Jan. 9, field agents were able to collect and send 181,280,466 new records collected from data-center networks, the documents said, ranging from metadata to actual content. The data-center surveillance and data collection remained secret even from Yahoo and Google. The effort was part of program code-named Windstop that was jointly supported by the NSA and a British spy agency known as the General Communications Headquarters (GCHQ).
The NSA tapped Yahoo and Google datacenters directly under a Windstop sub-project that used the cover name MUSCULAR and a network access point outside the United States that was identified only as DS-200B.
That network point provided a connection to an unnamed telecommunications company that allowed the NSA direct access to a router or switch through which Yahoo and Google datacenter traffic passed. With direct access to the networking hardware, the NSA would have been able to read, record, redirect, delete or modify traffic traveling between datacenter sites.
The NSA used custom-written demuxers to separate the firehose into separate data streams that could be processed automatically, after being routed to a storage system called Pinwale that had tools to automatically search and retrieve text in intercepted email and text messages.
In July 2012, the NSA discovered Yahoo was transferring thousands of email accounts to archive servers after converting them into a proprietary data format called Narchive. The NSA had to custom-develop demuxers to translate that data, though analysts at the agency’s Analysis and Production directorate asked that the Narchive collection be slowed down. Narchive data made up a quarter of all the information collected through MUSCULAR, but the accounts being transferred were at least six months older than other MUSCULAR data feeds, and the information was of low intelligence value, the Post reported.
The datacenter-tapping effort is separate from its effort to collect metadata and other online communications under programs allowed under the Foreign Intelligence Surveillance Act and supervised by the Foreign Intelligence Surveillance Court, the Post reported. The effort to tap Yahoo and Google datacenter networks directly is far more ambitious, resulted in the collection of a far higher volume of data, and appears to have been secret even from companies themselves. Tapping the direct links between datacenters made it easier to collect high volumes of data, and do it in close to real time.
To accomplish the task, the NSA had to find a hard connection between the private datacenter networks owned by Yahoo and Google and the public Internet. Then it had to defeat the encryption and security the two search companies used to protect their connections.
A sketch from the documents showing where the attack took place also included a smiley face celebrating the agency’s defeat of Google security.
In response to the revelations, Google issued a statement saying it is “troubled by allegations of the government intercepting traffic between our datacenters, and we are not aware of this activity,” adding: “We have long been concerned about the possibility of this kind of snooping, which is why we continue to extend encryption across more and more Google services and links.” Yahoo also denied knowing the NSA had been listening in.
The NSA declined to comment on the stories, according to the Post.
Image:NSA via Edward Snowden, via Washington Post