Big data security analytics still immature, say security experts

The big data security analytics concept and the tools are still immature, according to a panel of security experts

While big data security analytics promises to deliver great insights in the battle against cyber threats, the concept and the tools are still immature, according to a panel of security experts.

The US Computer Emergency Response Team (US-CERT) has yet to achieve its vision for big data security analytics, said Peter Fonash, CTO for the cyber security office at the US Department of Homeland Security.

“We are still working towards our vision of a cyber threat 'weather map' that is predictive,” he told the Global Cyber Security Innovation Summit in London.

An important goal for big data analytics is to enable organisations to identify unknown indicators of attack, and uncover things like when compromised credentials are being used to bypass defences.

At the recent Nato summit in Wales, CERT-UK was able to mine social media data to support operations to maintain cyber security at the event.

“However, handling unstructured data and combing it with structured data to arrive at an accurate assessment is one of the big challenges,” said Neil Cassidy, deputy director for operations in the UK's national Computer Emergency Response Team (CERT-UK).

“At the Nato conference, it was challenging to establish what claims were true and which were false to know what was actually happening,” he said.

Combining data from multiple sources

But suppliers said big data security analytics is already delivering value by enabling organisations to analyse data from previously disconnected security data sources.

“Attackers are exploiting the fact that security data is in silos,” said Feris Rifai, CEO of analytics firm Bay Dynamics.

“Security analytics is not only about big data repositories, it is also about collecting together lots of small bits of data from point solutions to make better decisions,” he said.

Rifai said the need for bid data security analytics has never been greater because IT security professionals spend most of their time on discovery.

“By looking at the intersections between data from multiple sources, security professionals can more quickly identify what they need to priortise,” he said.

Most organisations, however, are still tending to store data and forget about it, rather than running multiple queries against it, which is key, said Peter LaMontagne, CEO at another analytics firm, Novetta Solutions.

But Fonash said many firms lack people with the right skills in running queries across multiple data sources.

“Another common challenge is how to disseminate information in real time or near real time and in a machine-readable format for process automation,” he said.

Identifying useful threat intelligence data

To overcome some of these challenges, Cassidy said CERT-UK is looking at structured language for cyber threat intelligence information called Structured Threat Information eXpression (Stix).

“Getting information out is a challenge, but we believe Stix could be key to enabling different CERTs to share information at speed and scale,” he said.

CERT-UK is working with counterparts in the US and Australia to find ways of getting information to defenders quickly in a format that is useful.

Handling unstructured data and combing it with structured data to arrive at an accurate assessment is one of the big challenge of big data security analytics

Neil Cassidy, CERT-UK

“But there are several issues around automatically ingesting information, including how to ensure that recipients can trust information sources,” said Cassidy.

A common challenge for US-CERT, said Fonash, is navigating internal politics to get complete datasets from various government agencies.

For all organisations, it remains a challenge to identify the most relevant or useful threat intelligence feeds.

“Linking a threat to an IP address, for example, is not necessarily useful, as that IP address could represent hundreds of thousands of machines or users,” said Cassidy.

But few organisations have a mature understanding of big data security analytics, and fewer still are actively using it as part of their cyber security strategy in the UK, Cassidy told Computer Weekly.

For big data security analytics to be successful, he said, organisations first need to have a clear idea of exactly what they want to get out of it.

“Only once you know what you want to achieve can you begin evaluating which are the most relevant technologies and feeds,” said Cassidy.

Read more on Hackers and cybercrime prevention