Business Intelligence: the case for predictive analytics

The idea of having a tool that can help organisations forecast the future is an appealing one, especially when living through the uncertainty of today's recessionary climate.

The idea of having a tool that can help organisations forecast the future is an appealing one, especially when living through the uncertainty of today's recessionary climate. But predictive analytics technology, which aspires to do just that, has been around in one shape or form for years and, while in steady growth mode, has...

failed to set the world alight to date.

According to IDC, the global predictive analytics (PA) market grew 12.1% in 2008 to $1.5bn and is expected to grow at a compound annual rate of 7% over the next five years. This compares with the more commoditised end-user query, reporting and analytics (QR&A) sector, which was worth $6.3m in 2008 and grew by 10.3%.

The big difference between the two segments, meanwhile, is that QR&A involves working with historical data to identify trends or patterns, while PA takes large volumes of both historical and real-time information from different internal and third-party sources, puts it through a model, and predicts likely outcomes based on a range of causal factors.

The technology is used mostly by large data-rich enterprises such as financial services firms. The most common applications in this context are to analyse whether customers pose a potential credit or insurance risk and/or to establish whether they are attempting to undertake fraudulent activity. Prior to the credit crunch, PA tools were also used to improve customer acquisition rates, although the focus is now more on retaining the most profitable ones at the expense of those providing a lower return.

Other key sectors, however, include telcos, which generally employ the software to predict likely customer activity such as churn and/or payment rates, and pharmaceutical companies, which use it in areas such as drug discovery.

Retail and leisure companies also comprise other core markets. They invariably use such systems to try to forecast demand for given products and services at different times of the year based, for example, on seasonal factors such as holidays or the weather. Another use case involves predicting whether individual customers will be interested in specific special offers in order to maximise contact with call centre agents via cross- and up-selling activity.

Prime example

Center Parcs is one example of a PA user that benefited to the sum of £2m during the first year of implementation. The provider of holiday villages always used to undertake two bulk marketing mailings to its customers each year. But after rolling out DataDistilleries' PA tools in 2002 as part of an overhaul of its marketing function, it introduced more frequent campaigns targeted only at those customers it believed were most likely to respond.

In the first year alone, the move enabled the company to cut mailing costs by more than 50%, saving it £1m. It was also able to boost revenues by the same amount as a result of increasing occupancy rates and of selling guests a wider range of sports and leisure facilities.

Another organisation that has used PA offerings for almost a decade, however, is Procter & Gamble. It employs SPSS' software to help it better understand and predict customers' probable future buying behaviour in order to market new and existing brands to them more effectively.

As John Hagerty, a research fellow at AMR Research, explains: "Predictive analytics tends to be geared towards outward customer-facing activity. It's primarily used to drive additional revenue creation and sometimes for optimisation. But it's only used in pockets and not in back-office operations much."

This is because the software is complex and has traditionally been used by a specialist team of statisticians who are employed for specific purposes in a large enterprise context. An expert team is required because practitioners need to understand which algorithms to use, in what context, and on which data sets.

But key vendors such as SPSS and the SAS Institute have been trying to both make such tools easier to use and simpler to make changes to underlying data and programming models, which can currently take days. According to Alys Woodward, an IDC programme manager, however, there's still a long way to go.

"Advanced analytics tools are more expensive than QR&A ones, but they're not crazy. The market's not inhibited by price, but more by the fact that you really need to understand this stuff and people don't know what to do with it to take the next step. So users are doing great things with it, but it's only about 5% of what they could do," she explains.

Underlying information

Another issue is the need for underlying information, which is generally housed in a large data warehouse for analytics purposes, to be complete, consistently defined and accurate. But to achieve such a state generally requires massive data cleansing and integrity exercises, which involve the IT department and data owners working together to undertake what is often quite a manual task.

Hagerty warns: "Without good data, predictive analytics is useless because it's the bedrock that everything else sits on top of. You can build a fancy system on top of a rotten core, but the results will be garbage. So it's about ensuring that you have common data that's clean and defendable."

As to what the future holds, although predictive analytics is likely to remain somewhat of a niche technology into the medium-term, it could well start to broaden out in usage terms over time.

Woodward, for example, believes that OEM arrangements that SPSS has signed with QR&A vendors such as Business Objects and Cognos could lead the charge to a widespread embedding of the technology over the next couple of years, not only in QR&A tools but also in other applications such as CRM.

Hagerty, on the other hand, expects to see a "real flowering of analytics" in a general sense over the next three to five years, with predictive analytics playing a key part in this.

"Once the technology evolves and more use cases emerge that are valuable and cost-effective, barriers will start to fall. It's already started as the concept of analytics has increasingly taken hold, but it will definitely become a more accepted way for organisations to exploit their data more fully," he concludes.

Read more on Business applications