Information security is a big data issue

First-generation security information and event management (SIEM) products have been overwhelmed by the volume of data they now harvest

Companies have long moved from the Fort Knox approach to IT security, realising that simply placing big walls around critical data can never be enough to fully protect the network. The challenge has become one of intelligence – monitoring and analysing all the activity taking place across every element of the IT infrastructure...

to identify threats.

But as a result, businesses are generating terabytes of security-related data every day, placing a huge analysis and reporting burden on hard-pressed information security teams. This is exacerbated by increasing demands by regulators, compliance teams and auditors for proof that security controls are working.

First-generation security information and event management (SIEM) technologies have been overwhelmed by the sheer volume. Security of data has become a big data problem in itself.

Take financial services giant Barclays, for example.

The bank generates 44 billion security events per month – a figure set to reach 65 billion by the end of the year, according to Stephen Gailey, former group head of security services for Barclays.

The company has long since gone beyond the capabilities of its conventional SIEM system, Gailey told a gathering of IT security leaders at Computer Weekly’s CW500 Security Club.

“We ended up deploying a SIEM, and for a while it was a great solution. It brought in data from all our disparate sources and allowed me to build a security operations team and feed events to them. They were able to react in real time,” he said.

CW500 Security Club

The June 2013 meeting of the CW500 Security Club discussed the topic of big data and information security. The three speakers were:

Stephen Gailey, SplunkStephen Gailey, former group head of security services for Barclays, who now works for Splunk

Amar Singh, News InternationalAmar Singh, chief information security officer at News International

Jitender Aurora, GE CapitalJitender Arora, senior programme manager of security and risk at GE Capital Europe

But as people wanted to ask more analytical questions of the SIEM data, and as new technologies were added to the network, such as domain controllers and proxy servers, the data collected became less useful.

“Suddenly all this data became no good to me, I was just storing it. So we threw out the SIEM and about three years ago implemented a big data solution,” he said.

“That SIEM had ceased to be able to cope at about 500 million events per day. It was a struggle to bring in new data sources because I had to go back to the SIEM vendor every time, and we couldn’t query the retained data.”

As a result, Gailey implemented software from Splunk – a decision that proved so successful that, two months ago, he left Barclays and went to work as a product evangelist for the supplier.

Regulatory compliance

Barclays realised that using Splunk to analyse data in real time meant it no longer needed SIEM.

“We could bring in new data sources that we couldn’t use with SIEM. If you’re in a regulated environment you can’t throw a lot of this data away,” said Gailey.

Splunk has brought big benefits to the company’s regulatory compliance team. The bank has to prove that all its controls are effective – the investment banking division alone has 176 separate regulators it has to satisfy worldwide.

One of the fraud-related regulations recommends that traders take a mandatory two-week holiday every year during which they are not allowed to log in to any systems, to prevent them hiding any fraudulent activity that may be revealed in their absence.

For example, when rogue trader Jerome Kerviel caused €4.9bn of losses at French bank Société Générale, he never took a holiday because he always had to keep hiding his fraudulent trading.

Read more on big data and security

Barclays’ compliance teams need to prove traders are not logging in during such periods. The firm did not have a holiday booking system to check against, so Gailey used Splunk to analyse log-in data to identify in real time all people who did not log in for a two-week period, which could be cross-checked against relevant staff.

Considering that some traders would have to log in to potentially dozens of different systems, generating huge amounts of security events, Gailey said such a task would not have been possible using the old SIEM tools.

“We were able to answer that question in real time, something we would never have been able to do before,” he said.

Another system used at Barclays was FireEye, a tool to detect unidentified threats on the network. Combining FireEye outputs with Splunk allowed Gailey’s team to highlight a number of previously unknown problems without having to purchase large numbers of FireEye devices.

“Apart from the obvious security challenge, Barclays used big data to help with compliance, audit and regulation. Information security teams are being asked to do a lot more than they ever have - it’s not just about configuring firewalls and so on, it’s about ticking all the compliance boxes. Getting compliance right is a big thing in a regulated environment. We have moved into a world where it’s not enough to have security controls, you have to demonstrate they are ubiquitous and they work,” said Gailey.

Security is now a big data problem because the data that has a security context is huge. It’s not just a collection of security tools producing data, it’s your whole organisation. If you’re going to ignore some of that data, or if you can’t analyse it, then you are not doing security properly. Every little thing you miss or ignore might make the difference to your company.”

Breach detection

Amar Singh, chief information security officer (CISO) at News International, told CW500 guests that security event analytics is a vital tool in improving detection of breaches.

“The point is to understand is what is normal, to know what is not,” he said.

People think that having more and more logs give us more insight. I don’t believe that’s the right concept. It’s not about big data, it’s about relevant data

Jitender Arora, GE Capital

Research suggests most security breaches are detected by third parties, not by the affected company itself – yet in 84% of breaches system logs were available to discover if a breach was taking place, said Singh.

“A lot of [IT security] is still check-box driven. It is reactive,” he said.

“For true visibility you need advanced analytics. You need the skills and the people who can give you that.”

A key task is to define what is normal for your organisation, Singh said. This covers many areas, such as user identity management, asset classification, threat intelligence, as well as information to give context to security events.

“What comes out of this are reports, alerts and intelligence about what is happening in your organisation, which helps to define normal,” he said.

“You can then identify users that are behaving outside of the norm. If they are identified early on their access can be disabled and potentially a breach stopped.”

But Jitender Arora, senior programme manager of security and risk at GE Capital Europe, warns against allowing the buzzword and hype around big data to take the focus away from the core principles of risk management.

“Data is just data. It doesn’t tell me anything,” he said.

“What I’m interested in is analysing data to come up with meaningful information that can tell me how to improve the situation. If data is not in the right business context, it can be completely irrelevant.”

About the CW500 Security Club

The CW500 Security Club is an exclusive networking club for information security leaders. It meets three times a year to offer peer-to-peer debate and sharing experiences around topical IT security issues. 

Highlights of recent CW500 Security Club events:

Arora said the huge volume of security data, combined with new big data tools such as Hadoop, can lead to a loss of discipline over managing that data because people assume they can store it all and then make use of it later.

“People think that having more and more logs give us more insight. I don’t believe that’s the right concept. It’s not about big data, it’s about relevant data,” he said.

“Big data is just sold as the next big thing. Every time we get a new buzzword, people think it is going to come along and solve all their problems. Unfortunately, I don’t think so.”

Context is everything

Arora cited the example of Hurricane Sandy, which wreaked havoc along parts of the north-east US coast last year. Some 20 million tweets were written on Twitter about the disaster, peaking before and after the passing of the hurricane.

But subsequent analysis showed that the majority of tweets originated in Manhattan, which was not threatened by the storm. Very few of the tweets actually came from the affected areas.

“If an emergency response team was using that data to help plan their activity, they would have got it wrong – they missed the context,” said Arora.

“Not every type of data will give you all the insights you need. The future is about having the right data analytics capability.”

Another myth is that big data will make companies more proactive in managing security, said Arora. Prioritising data analysis based on business need is more important, he said: “All it can do is make us react much faster. Analysts can generate reports faster and understand events faster, as well as help in forensic examinations. If you think that by implementing big data it will make you proactive, it’s not going to happen. It’s not about big, but about relevant data.”

But Gailey concluded that a big data approach will help companies to gain a better understanding of the scale of modern information security challenges.

“I don’t think anybody is estimating the real cost of data breaches. Organisations either don’t know or are very bad at estimating,” he said.

“Is security a big data problem? It is, because there is a big amount of data out there with security in it that you need to analyse.”

Read more on IT risk management