In many ways, big data analysis concerns finding hidden unknowns in a large data set. Tibco is applying big data techniques to event processing.
In an interview at the Tibco Transform conference in London Matt Quinn, CTO of Tibco, says: “You have lots of rich historical information and we have tools that can extract patterns of events from all that information, such as a pattern of activity that is indicative of fraud."
Tibco calls this type of analysis the “two-second advantage”. He said: “The two-second advantage means having a little bit of the right information at the right time can make more of a difference than having much more information six months down the line.”
According to Quinn, the trick is to look at transactions and find a common pattern such as the attributes that would indicate a fraudulent transaction. The Tibco vision concerns looking at transactions using its event-processing technology to link patterns of activity in a stream of transactions.
He said the event stream analysis technology can be used to apply fraud detection in real time to look for patterns that would indicate fraud as they emerge.
“This gives you the advantage of anticipation. You've seen a pattern before and it is starting to look like fraud. You can stop it in real time, rather than wait until you run a loss report in six months' time," he said.
Tibco's customers are using the company’s event processing technology with its Spotify analytics tool to implement real-time pattern matching.
It is not always necessary to look at historical data in order to build patterns to analyse in an event stream. “Sometimes you know what the rules and patterns are. But because people are starting to look at large volumes of data, they are finding new patterns that perhaps were not obvious to the domain expert,” Quinn said.