Gathering retail intelligence

Understanding the customer

The task of using business intelligence to inform decisions is crucial for bricks-and-mortar retailers who are fending off online-only retailers from cannibalising the high street. But the scale of that task is also more complicated, as high-street retailers have multiple legacy systems and a more anonymous customer base compared with onlineonly sellers. “Online retailers have the advantage with customer data, as customers can log in and identify themselves to purchase online, making the experience automatically more tailored to them,” says Bjorn Weber, analyst at Planet Retail. “They are data players and already likely to be experts if successful in the market.”

Weber says that customer use of tablets and smartphones is driving big changes in retail technology and business intelligence systems.

“There is a new need for speed in the context of data and information in the retail sphere,” he says. “Non-food retailers are losing business to e-commerce companies and need to rapidly change the systems available to them so that customers don’t have better information than they do.

We are also examining some of the new data warehouse platforms that have the capability of allowing us to analyse better

Neil McGowan, CIO of JD Williams

“Retailers need to enable and interact in a clever way with the shopper and their smartphone and tablet PCs, but also enable their own staff to support the shopper,” says Weber. Much of this will come from joining up information about the customer online with the in-store experience. For example, retailers could offer shoppers an in-store discount on certain items via their smartphones. Such incentives could even be personalised to the customer.

“Using real-time data accessibility to make quicker decisions on pricing and product information, enabling clever discounting, is becoming very important,” he says. In principle, that would allow retailers to react faster on price trends, and discount things accordingly, but at the moment the systems are so complex that buyers cannot see the whole truth, Weber says.

Making decisions quicker

But Weber believes the technology to enable that transformation is not quite there. “We need real-time data warehouses. The business intelligence architecture is still far from being fast enough to support the decision-makers and store manager in real-time. “Most of the people I speak with say it is the database systems that are too slow. The most important data is that being scanned from the checkout,” he says.

SAP is one of the big software providers to the retail sector, making a lot of noise about accelerated data analytics, thanks to its inmemory database, Hana. But actual applications such as forecast engines are not yet available, according to Weber. Similarly, Oracle is still selling traditional database technology to retailers. And, of course, even though a retailer can track stock levels accurately from the point of sale, once the stock has gone, there is very little a retailer can do.

However, Weber says there are still big developments occurring in business intelligence tools for retailers. Replenishment automation has had the biggest impact on retailers over the past 10 years, reducing inventory by around 10%, he says. But now that technology is moving into other areas, such as assortment and price optimisation, he says.

Assortment optimisation enables retailers to predict which lines should be delisted, enabling them to be more responsive to changing consumer preferences, while price optimisation can gain the edge over competitors.

“If retailers are looking to improve their price image, they now have access to really sophisticated software that can allow them to do that. Those tools use massive data warehouses,” says Weber.

From catalogue to online

Neil McGowan, CIO of online plus-size shopping company JD Williams, has overseen the firm’s shift from being a catalogue business to an online retailer, and the information issues that have occurred with that journey.

“We have an advantage as a traditional direct retailer, as we already have a strong confidence in data analysis and analysing datasets based on the catalogue and call centre model,” he says.

“There is an awful lot of data to capture, and deciding how to filter that information to make it relevant is a challenge,” he says. Intelligent data mining and the use of real-time information is key to the company’s success. “If someone comes onto the site and we know who they are and that they haven’t been to the website for a while, we might decide to give them an extra discount as an incentive. We can filter offers accordingly, using an embedded cookie on the user’s permission. Much like the way Amazon operates,” he says. Ultimately, effective use of business intelligence can create a more personalised shopping experience.

“We do have a wide product range, which can create the impression of being a cluttered shop. So the goal is to filter out certain products and make it relevant to what the particular customer is interested in. That way we can create a personal experience relevant to what the customers want to browse.

“Offline, the analysis is about what business can do to understand different groupings, and trends coming out of [the buying information]. The challenge is how to aggregate that information. It is a big data challenge,” says McGowan. “We are also looking at more modern approaches, such as Hadoop, that allow us to analyse less structured datasets. We’re just starting to do a trial in those areas using IBM’s Netezza analytics, which is modern, used in big data approaches and has Hadoop at its core.”  The availability of real-time information is another key opportunity, he says. “So for example you might be able to quickly provide a personalised offer to someone on trousers if they are only buying shirts.”

According to McGowan, real-time web analysis is not too difficult to achieve using embedded Java code on the website. Such information can be analysed using a data warehouse engine like Teradata.

“We are also examining some of the new data warehouse platforms that have the capability of allowing us to analyse better,” he says.

Making sense of tweets Social media has dramatically changed the way retailers gather information, as customers are now in the habit of commenting on products and contacting companies directly through channels such as Twitter.

Tools such as Radian6 from Salesforce.com allow retailers to measure customer sentiment to gain a better understanding of their customers.

But the main issue is that most of the data coming from social media is unstructured and comes in a format where consumers are not necessarily using literal meanings – such as the use of “bad”, says Christine Bardwell, research manager at analyst firm IDC. “So the issue is how do you cut through that, gain sentiment and understand what that means.” The creation of a “single view” of the customer is now becoming key, says Bardwell.

“The most effective way of doing that is a master data management (MDM) system that de-duplicates information about the customer. This could create a single line about the customer that may hold many data points such as transactions, what was bought, and information on Facebook.

“How it looks in the database is irrelevant – it’s important there is just one profile and then all other systems feed in and out, such as online systems, in-store systems, and loyalty schemes that can then be updated.”

Investments in integration are likely to continue, as traditional retailers tackle multiple legacy systems, often described as spaghetti because there are so many of them. “Retailers wanted to move quickly on omni-channel and increase customer loyalty, but mostly there has not yet been a long-term view – just a lot of linking up in the background,” says Bardwell. “There is not a long-term view about the foundational layer. MDM will be a big investment for retailers,” she says.

“Linking up the order management and inventory has been the priority, that has been the main focus to date. Customer and analytics comes further along the omni-channel journey.