Barclays indexes machine data to meet complex regulation

Barclays turns to machine data indexing Splunk to deal with complex banking regulations following the financial crisis of 2008

Barclays has turned to machine data indexing Splunk technology to deal with the demands of the more complex regulatory banking environment created following the financial crises of the 2008 and aftermath.

Stephen Gailey is EMEA director of financial services at Splunk. He joined from Barclays, where he was group head of security services.

Monitoring the logs from security technologies, such as intrusion detection systems, is, confirmed Gailey, one of the 15 or so uses for the technology either in practice at Barclays or under consideration.

The bank generates a least two petabytes of data every day, Gailey discovered, so it has a lot of machine data to index.

His group first engaged with Splunk in 2010, eventually signing a licence deal for 2TB per day. Up until then, the bank had used a security information and event management system from Intellitactics, acquired by Trustwave in 2010.

“We were running into the limitations of the technology,” Gailey said. “It would not scale, and the regulators are asking for increasingly detailed reports on the [machine] data collected. We found that the SIM [security information management] was poor at asking complex questions, so we tried running a log management system, from LogLogic, alongside it”.

Gailey and his team decided to cast this system aside in favour of Splunk, despite having secured senior management investment for it.

“That felt like a brave decision, but if I hadn’t done that I would have been in a difficult position a year on. The hard part was convincing the retail bank to join in with the investment side,” Gailey said.. But they were convinced.

Read more about machine data used in banking

The original security use case of the technology generating alerts for what was Gailey’s team is still in operation. But it is also being used to provide analytics for the more general operation of security controls. “It is the single plane of glass that tells us about the risk status in the bank, across many point solutions, dealing with internal and external threats,” Gailey said.

The technology is also either in use or under consideration at the bank for high performance computing, used in calculating risk, and high speed trading.

On the retail side of Barclays, Splunk is used in the architecture for Pingit, the bank’s mobile app.

Gailey said the technology has proved itself a good return on investment. Its deployment was offset as a cost, in part, by not buying the log management system and decommissioning standalone SIM technologies. But there was an operational benefit, too, he said, in that the Splunk technology automatically provides context around alerts that would previously had required lengthier investigation. And it meant saving on training.

“We would only have to avoid one fine for it to pay for itself, since such fines are in the millions of dollars. It has made compliance much easier,” Gailey said. One example of that was a Monetary Audit of Singapore [MAS] audit. Having the privileged access logs in Splunk made that easier, he said.

Read more on Big data analytics