momius - Fotolia

Gartner on the Pentagon’s ‘misconfigured’ AWS S3 bucket data leak

After details of the Pentagon’s online data collection habits came to light in the wake of its recent AWS S3 bucket leak, Gartner said web users should not be surprised to know the intelligence services are keeping tabs on their online activities

Internet users should not be surprised the Pentagon has been collecting and storing their online posts, according to Gartner vice-president and distinguished analyst Avivah Litan.

This comes after a cyber resilience researcher uncovered information, suggesting the organisation has been collecting social media posts from web users across the globe for eight or so years, stored in Amazon Web Service’s (AWS) Simple Storage Service buckets. 

In a blog post dated 17 November, Chris Vickery, risk researcher at cyber resilience company UpGuard, documented the discovery of billions of online posts and news commentary in three AWS S3 buckets that were publicly accessible. 

Computer Weekly understands the leak is being attributed to an unintentional misconfiguration of the AWS S3 cloud storage system on the user side.

AWS recently introduced default encryption for S3 in the wake of other incidents whereby misconfiguration errors have resulted in companies accidentally leaking data. 

The collected data is said to have loose correlations to US security concerns, with posts containing details about Iraqi and Pakistani politics, which Vickery said raises serious questions about the privacy and civil liberties of citizens.

Speaking to Computer Weekly, Litan said the revelation should not come as a surprise to people, given these methods are routinely used to identify dangerous individuals.

“It’s a free for all on the public internet, and intelligence agencies can justifiably use public data and postings to detect criminal activity and security threats against national interests,” she said.

Read more about AWS

“Unfortunately, innocent peoples’ data and postings are by default swept up in these massive data gathering exercises, but data from ‘good’ and ‘bad’ entities are required to develop models to detect the ‘bad’.

“Public data on theinternet can be extremely useful for finding criminals, terrorists, malicious nation state actors, and in pre-empting or blocking potentially devastating attacks conducted by these entities,” she said.

The Pentagon play down

The Pentagon played down the sensitivity of the released information, with Central Command spokesperson Earl Brown making the point that the released information is already publicly available.

“It is not collected nor processed for any intelligence purposes. All of the information is readily available public information related to our activities and obtained through commercial off-the-shelf programmes in accordance with US Code and Department of Defense policy in a consistent manner,” he said.

“US Central Command has used commercial off-the-shelf and web-based programmes to support public information gathering, measurement and engagement activities of our online programmes on public sites.

“The information is widely available to anyone who conducts similar online activities. The data is raw data that was provided to us by a contractor,” he added.

Gartner analyst Litan also said the worrying aspect of this Pentagon discovery was that the repositories were not locked down: “The concerning piece of this leak is the lack of care that the information gatherers gave to securing their repositories that turned seemingly public data into meaningful sensitive information.”

Read more on Infrastructure-as-a-Service (IaaS)