Get real value from real-time systems

A few years ago, everyone was talking about the real-time enterprise and how it was destined to be the future. The idea was that companies would be instantaneously aware of any important business events taking place so that they could respond appropriately and immediately.

A few years ago, everyone was talking about the real-time enterprise and how it was destined to be the future. The idea was that companies would be instantaneously aware of any important business events taking place so that they could respond appropriately and immediately.

But that has not happened because of both the cost and complexity of such a proposition. As a result, the hype has moved on to real-time data analytics for certain sets of data rather than for all of it.

This area is, in turn, facing its own problems. Although the concept of being able to make snap business decisions based on up-to-the minute information may resonate with a lot of people, too many are confused as to what it means and how they can make it a reality.

What real-time means

And one of the issues here is simply that of terminology. Gareth Herschel, a research director at Gartner, says, "For most organisations, data is not truly real-time. The term is a bit misleading and it is a bit of a false goal for organisations to work to.

"The question is, is there a real benefit in being able to analyse data minute-by-minute or is day-by-day or week-by-week enough? And the answer really depends on how fast their market moves."

Herschel says the concept of "right time and right latency" is more apt. "The more real time data is, the better, but the closer and closer it gets to that, the more systems cost to put in place. So there has to be a clear business benefit for investment," he says.

Steve Shepherd, solutions sales manager at IT services provider Logicalis, says that organisations are unlikely to get much change out of £500,000 for a real-time data analysis system once software licences and services are taken into account.

"These things are not a small investment. If you just put a glass sheet over your systems and introduce a notification tool to say that 'based on these conditions, I will send you a report', it will not cost too much. But if you are trying to profile a customer using data from different systems to do more in-depth analysis, it becomes much more involved and, therefore, much more expensive," he says.

Real-time reporting systems

And this highlights another problem - of what exactly real-time data analytics means. Herschel says there are three key categories of system, all of which are at varying stages of maturity. The first area comprises reporting, but this is rarely undertaken in true real-time.

"Most enterprises are trying to lower the latency of how frequently they update and collect data from across the organisation. This tends to be a knock-on effect from investing in a data warehouse and so is quite a universal trend," he says.

Business activity monitoring

The second field is business activity monitoring (BAM) and complex event processing (CEP), with the former being a sub-set of the latter.

With BAM, the idea is that organisations receive alerts when a specific but unusual event takes place so they are in a position to respond quickly. An example might be a package being delayed at customs or a customer cheque bouncing.

CEP, however, is about companies being able to respond swiftly to multiple anomalous events. So, for example, if a customer did not receive their salary one month but deposited a large one-off payment the next, their bank might deduce that they had lost their job and just received a severance payment. This means that, although it might not be a good time to try to sell them a mortgage, it might be appropriate to offer them a loan.

"Every company could probably benefit from this in specific areas of their business because it is about notifying people of problems before they become significant so that action can be taken. But, although some enterprises have been doing it for a while, it is relatively early days," Herschel says.

Real-time data mining

The third category is real-time data mining. The goal here is to provide guidance to call centre agents, for example, about what action to take at a given point in the sales or service process.

So, if a customer phones to pay a bill the system might recommend that the call centre agent promote a specific product or even nothing at all based on an up-to-the-minute profile.

"This is one of the hottest areas of customer relationship management (CRM) and is an area that call centres in sectors such as telecoms, financial services, retail, and travel and leisure are looking at.

"It has been around for about 10 years as a concept, but few companies are using it, because it requires a change of mindset on the part of the organisation," says Herschel.

A key challenge here is that the performance of most call centres is measured on how quickly they answer customer calls. Cross-selling and up-selling slows this process down.

Therefore, organisations often find they need to hire additional staff, which will need to be trained in sales. Existing personnel may also become unhappy with their new role, which leads to attrition.

"Real time analytics projects are very similar to CRM projects in that most are not technical failures, but organisational ones. There is no point having information at your fingertips if it takes a week to react to it. So it is all about the decision that you are trying to make," Herschel says.

Case study: Zavvi Entertainment

Zavvi Entertainment Group, which bought out Virgin Retail, went live with a near real-time system about 14 months ago, with the aim of enabling stores to provide customers with up-to-date information about stock availability.

Another key goal was to enable central buyers to view, at 15-minute intervals, how well or badly individual products were selling in order to improve purchasing decisions.

For Tony Johnson, IT director at the Zavvi Entertainment Group, faster access to relevant information has a definite business benefit.

"Getting early visibility of day one sales and being able to respond does give us a distinct commercial advantage, especially with new [computer] games, which are always released on a Friday and quite often have a constrained supply," says Johnson.

Previously, because Zavvi used overnight batch processing to update information, the first time company analysts had a chance to look at the day one sales data was on Monday morning.

"But now we can look at reports at any point on Friday so that we can make decisions rapidly. This gives our central buyers more chance of procuring popular products, which has a direct impact on revenues," Johnson says.

Johnson says that Zavvi decided to update its data at 15-minute intervals rather than on a real-time basis for pragmatic reasons.

"Initially, we were not 100% sure that the system would cope with more regular updates, because you have to take account of the overhead on store systems, the data warehouse, the reporting and analysis tools and on pulling data over the network. We have since found that we could do it significantly more frequently, but we have also found that we do not need to," he says.

An issues to bear in mind here is that if information is updated too regularly staff can "spend their lives in front of the screen pressing the refresh button", particularly at first because of the novelty factor.

But Johnson also says, "It is absolutely fundamental to be clear about why you want to introduce real-time data analysis and the use that the data is going to be put to. It is great for people to look at it and say we have done this volume of sales at this time, but if they are not acting on that data, it could end up just being a distraction."

He also says that it is sensible to start small with one or two clear goals in order to prove that the system works, while at the same time building on what is in place "to exploit existing investments".

Zavvi's had already implemented a data warehouse along with MicroStrategy's online analytical processing tool, but subsequently deployed Microsoft's BizTalk middleware to act as the glue between key systems - and to provide an interface to suppliers' systems in order to improve data exchange.

"BizTalk pulls data from the store point-of-sale systems back to the central data warehouse and a heavy amount of work went into developing the interfaces here. Significant work was also needed to build the MicroStrategy reports to deliver information to the buyers' desktops and to push that data into our merchandising system," says Johnson.

Nonetheless, Zavvi managed to implement its new system in six months.

Case study: Unilever

The same was true of Unilever, which introduced a near real-time data reporting system from IRI Research at one of its Southern European subsidiaries this year. It also has plans to roll the offering out across several others, including the UK, in the year ahead.

The aim, says Peter Kollecker, customer development process manager at the company's development process office Europe, is to analyse sales data from each of its retailers three or four times a day "to see where things are at with on-shelf availability, how new products are doing in the first four weeks of introduction and whether products are selling fast, slow or not at all following promotions".

The underlying objective is to enable Unilever to react more quickly to market conditions and to be more proactive in its dealings with retailers in order to meet their stock requirements more effectively.

But a key issue here is one of building up trust between the two parties. "There has to be trust there because you want to expose data to each other so communication is very important.

"We have been very open about what we want to report on and the outcomes we want, but it is also important to explain what benefits there will be to both sides. It has to be a win-win situation," says Kollecker."

Read more on Business intelligence software