The US Computer Emergency Response Team (US-CERT) has yet to achieve its vision for big data security analytics, said Peter Fonash, CTO for the cyber security office at the US Department of Homeland Security.
“We are still working towards our vision of a cyber threat 'weather map' that is predictive,” he told the Global Cyber Security Innovation Summit in London.
An important goal for big data analytics is to enable organisations to identify unknown indicators of attack, and uncover things like when compromised credentials are being used to bypass defences.
“However, handling unstructured data and combing it with structured data to arrive at an accurate assessment is one of the big challenges,” said Neil Cassidy, deputy director for operations in the UK's national Computer Emergency Response Team (CERT-UK).
“At the Nato conference, it was challenging to establish what claims were true and which were false to know what was actually happening,” he said.
READ MORE ON DATA ANALYTICS
- Sharpen your data analytics strategy
- Analytics & Data Warehousing Reader Survey Europe
- Jeanne Harris: Successful business analytics combines data and intuition
- HP Autonomy executive: big data transcends analytics
- Big data and analytics drive PC server sales
- Big data storage: Analytics takes small first steps in Turkey
- Big data security analytics: Facebook's ThreatData framework
- UK lags France and Germany in big data analytics, but sees itself ahead
- Big data analytics projects raise stakes for predictive models
Combining data from multiple sources
But suppliers said big data security analytics is already delivering value by enabling organisations to analyse data from previously disconnected security data sources.
“Attackers are exploiting the fact that security data is in silos,” said Feris Rifai, CEO of analytics firm Bay Dynamics.
“Security analytics is not only about big data repositories, it is also about collecting together lots of small bits of data from point solutions to make better decisions,” he said.
Rifai said the need for bid data security analytics has never been greater because IT security professionals spend most of their time on discovery.
“By looking at the intersections between data from multiple sources, security professionals can more quickly identify what they need to priortise,” he said.
Most organisations, however, are still tending to store data and forget about it, rather than running multiple queries against it, which is key, said Peter LaMontagne, CEO at another analytics firm, Novetta Solutions.
But Fonash said many firms lack people with the right skills in running queries across multiple data sources.
“Another common challenge is how to disseminate information in real time or near real time and in a machine-readable format for process automation,” he said.
Identifying useful threat intelligence data
To overcome some of these challenges, Cassidy said CERT-UK is looking at structured language for cyber threat intelligence information called Structured Threat Information eXpression (Stix).
“Getting information out is a challenge, but we believe Stix could be key to enabling different CERTs to share information at speed and scale,” he said.
CERT-UK is working with counterparts in the US and Australia to find ways of getting information to defenders quickly in a format that is useful.
Handling unstructured data and combing it with structured data to arrive at an accurate assessment is one of the big challenge of big data security analytics
Neil Cassidy, CERT-UK
“But there are several issues around automatically ingesting information, including how to ensure that recipients can trust information sources,” said Cassidy.
A common challenge for US-CERT, said Fonash, is navigating internal politics to get complete datasets from various government agencies.
For all organisations, it remains a challenge to identify the most relevant or useful threat intelligence feeds.
“Linking a threat to an IP address, for example, is not necessarily useful, as that IP address could represent hundreds of thousands of machines or users,” said Cassidy.
But few organisations have a mature understanding of big data security analytics, and fewer still are actively using it as part of their cyber security strategy in the UK, Cassidy told Computer Weekly.
For big data security analytics to be successful, he said, organisations first need to have a clear idea of exactly what they want to get out of it.
“Only once you know what you want to achieve can you begin evaluating which are the most relevant technologies and feeds,” said Cassidy.
Read more on Hackers and cybercrime prevention
How an idea built on web freebies and low-cost tools evolved into a virtual NHS clinic
Xerox report: SMB print requirements will shift in 2017
At health IT conferences, information blocking, telemedicine star
US Vice President Joe Biden shares his information blocking experience at Health Datapalooza