The National Security Agency (NSA) failed to detect the relatively unsophisticated data mining activities of whistleblower Edward Snowden, it has emerged.
Investigators have found Snowden used web crawlers to access and copy about 1.7 million documents, according to the New York Times (NYT).
The software is widely available and designed to search, index and back up a website, but was used by Snowden to harvest classified data.
NSA officials declined to say which web crawler Snowden had used, or whether he had written some of the software himself.
The news has deepened concerns that Snowden was not discovered and stopped in the light of the fact that the NSA is responsible for defending sensitive US computer systems from cyber attack.
Snowden’s insider attack, by contrast, was hardly sophisticated and should have been easily detected, investigators said.
Read more about the insider threat
- Privileged accounts key to most APT attacks, says Cyber-Ark
- Stopping privilege creep: Limiting user privileges with access reviews
- Privileged user management a must for DBAs
- Privileged account policy: Securely managing privileged accounts
- Privileged accounts are hacker sweet spot
- Privilege access management: User account provisioning best practices
- Security Think Tank: Least privilege is key to blocking IP theft
- Intel CPU hardware vulnerable to a privilege escalation attack
- Windows security case study: Controlling Windows 7 user privileges
- Exchange Server administration policy: Managing privileged user access
Almost three years earlier, a similar technique was used to harvest data from the US State Department, that was passed on to WikiLeaks.
Snowden had broad access to the NSA’s complete files because he was working as a technology contractor for the agency in Hawaii, helping to manage the agency’s computer systems.
An NSA official told the NYT that Snowden had been “challenged a few times” but he had been able to persuade investigators his actions were in line with his work as a system administrator.
Investigators say Snowden exploited the fact that, while the NSA had built enormously high electronic barriers to keep out foreign invaders, it had rudimentary protections against insiders.
He also exploited the fact that he was working at an NSA outpost in Hawaii that had yet to be equipped with modern monitors, which might have sounded the alarm.
Insider threats go unaddressed
In October 2013, a survey revealed that large enterprises are not doing enough to detect and address insider threats.
The survey of more than 700 IT security decision-makers found that less than a third of respondents said they block privileged user access to data to mitigate insider attacks.
However, the study also showed attitudes changing, with 45% saying that Snowden’s revelations about US internet surveillance has caused them to be more aware of insider threats.
Some 78% said they were either using or planning to use data encryption and 70% said they were using or planning to use data access controls.