Big data technologies lend themselves to unintended use cases.
Godfrey Sullivan, president and chief executive officer of machine-generated data indexing company Splunk, estimates that 30% to 40% of the company’s revenue comes from uses it did not envisage.
Sullivan was the president and CEO of Hyperion, the business intelligence company acquired by Oracle in 2007, where he says none of the company's revenue derived from unintended use cases.
That was unsurprising, he says, given that Hyperion supports the financial planning cycle workflow, guided by consistent accounting rules. “Splunk strives for discovery, Hyperion for consistency,” he says.
Sullivan cites a Japanese company’s indexing and analysis of elevator data, which disclosed that foot traffic was a leading economic indicator of lease renewals. Another customer, in Europe, was using Splunk to index data from electric car charging stations, but also discovered it was massively exceeding its mobile phone bills.
“I would say that Splunk was the first meaningful technology for this whole new generation of data, called big data. It does help to be first mover. It was a slogan without a solution, but machine data is what is relevant about big data," he says.
More on machine data
- Splunk shops find surprising uses for machine-generated data
- What is machine data?
- Loggly CEO: Log data management takes on 'the original big data'
Devices and the big data explosion
“The coming of age of the internet, transactions over the web, the rise of mobile, and all the devices being able to talk to each other are the conditions for the big data explosion. It is device data that is the single biggest cause of this,” he says.
Splunk has coined the term "operational intelligence" to use as a category definition, like business intelligence (BI) or customer relationship management (CRM). "To us it defines the outcome from ingesting machine data and then learning from that and gaining business value," he says.
“BI is the world of structured data; ours is the world of semi-structured and unstructured data. So, what you looked at [on an e-commerce site] and did not buy is collected as a data point in a way that is not true for the physical shopping experience,” he contends. “What iPhone? What version? are the sorts of questions answered by machine data.”
Sullivan says Splunk’s technology is especially good at real-time data analytics. “Time is the most important dimension of machine data, being able to understand trends over time, and so on”.
The company has a "save as Hadoop" function, and is not a data store as such, he says, but rather an analytical engine. "Hadoop is great for cheap batch storage.”
Sharing technical wizardry throughout the business
Splunk technology’s ability to index and render searchable machine data is similar to what Google has done with web documents, according to Sullivan.
It started off in security and information event management, and was categorised that way by the analyst firms. But the business now breaks down more broadly, he says. “It is not just about IT troubleshooting, but rather business value.”
More on big data
He confirms the breakdown is 25% security, 35% applications management, 30% physical and virtual infrastructure management, and 10% analytics. The latter is increasing faster, but has come later, he says.
The company has 1,300 customers in Europe. One of these is Barclays, which Sullivan says is ingesting 450 machine event types in its Splunk index.
But big data technologies, such as his own company’s, are “still at early evangelism stage”, according to Sullivan. And he reports that CIOs in the UK, whose firms are Splunk customers in the US, still ask, What can I do with Splunk?
“Executives in all companies are out of touch at the individual contributor level. We don’t want to be, or mean to be, but it is how it is," he says.
"So you need to ask the guys [at rank and file level]. We try to unify the CIOs with the guys who understand the technology, two or three levels away. The technical wizards know the technology; they just need an audience.”