Cut big data down to size

Big data value comes from interactive discovery not possible with standard business intelligence implementations, says HCL EAS

Businesses are processing more and greater varieties of data than ever before, which is being driven by advances in technology and connectivity. All this data provides organisations with greater opportunities to improve customer service, increase operational efficiency and create new products/markets. 

The volume, velocity, variety and variability of big data make it very different to traditional data analysis. The types of data processing involved and the level at which it needs to be done requires a well thought out approach. Ultimately, the organisations that are able to grasp this are likely to be the future winners.

Getting value from big data

Value is the one aspect of big data that most organisations are struggling with. The excitement surrounding big data needs to be translated quickly into business value to justify the considerable investments required. 

With big data, value is generated by interactive discovery, finding the golden nuggets of data through making associations, correlations and extrapolations that would not be possible with standard business intelligence (BI) implementations. Typically these fall into three categories:

1. Retrospective insight

Retrospective insight is the key to predictive analytics. In plain terms it can be thought of as learning from the past. For example, while equipment is typically maintained using a schedule, it rarely fails on a schedule. As a result, organisations can spend money maintaining equipment unnecessarily or incur extra costs due to unexpected failures. 

By analysing the historical condition of equipment, performance and failure information, organisations can start to accurately diagnose potential failures and increase the economic effectiveness of their maintenance expenditure.

2. Predictive analytics

Predictive analytics use historical and current information to predict future events and they can be used throughout an organisation, from improving business processes to modelling different business strategies.

For example, traditional call centre screens follow an 80:20 rule, cramming enough information on the screen to enable agents to handle 80% of the calls without moving to another screen.

For more on getting good stuff from big data

  • Beyond the hype, CIOs can generate business value from big data tools
  • The value of big data puts PayPal in pursuit of 'Analytics 3.0'
  • Enterprise search software helps users uncover value in 'big data'

3. Real-timeoptimisation

As individuals, even when doing the simplest tasks, we constantly process a wide variety of information. What tasks do I need to get done before I leave? What tasks can I do from home? What is the weather like now, and what is the forecast in the next few hours? Will the traffic be busier than normal? Do I need to put petrol in my car, and if so when?

Organisations are not nearly so sophisticated and certainly nowhere near as close to real time. The ability to efficiently process large volumes of data in real time, when combined with an ever-increasing network of sensors (RFID, GPS, Scada systems, telemetry systems, smart meters) is laying the foundation for "business decisions in real time".

Big data, and the step change in processing capabilities it provides, has the power to make organisations act more like humans who constantly seek to manage trade-offs and optimise rather than wait for the end of the month to figure out what went wrong.

Developing a big data strategy

It is clear that big data has the potential to deliver significant value to an organisation, but first it needs to create a clear strategy. 

Gartner recently reported that only one-third of businesses are considering the implementation of a big data strategy. Organisations need to learn to walk before they can run when it comes to big data. To do this effectively, organisations need to fully understand what is possible with big data and how it can help them achieve their corporate goals.

Businesses need to start on a small scale when approaching big data, as small achievements can deliver real value that will build confidence and support. Large failures will stop any strategy in its tracks, therefore once organisations see the benefits of starting on a small scale their confidence will grow.

Businesses must also identify professionals who can convert information into intelligence. A team of professionals whose sole focus is analysing data will enable organisations to have an effective big data strategy. 

Gartner recently reported that only one-third of businesses are confident of executing a big data strategy with their existing staff. As a result, businesses must realise that it is as important to acquire the right professionals as it is investing in big data technology.

Another challenge for businesses is to ensure that they are using the right technology. Big data initiatives differ significantly from traditional BI and DW implementations. Often they require a different way of thinking when it comes to the underlying technology architecture. 

As one example, SAP Business Suite on Hana promises to reduce total cost of ownership and deliver efficiency savings to users by enabling them to do things faster and at a greater volume. 

Unfortunately, it can be all too easy for organisations to lose sight of their original big data strategy. Therefore, to keep projects on track it is essential that organisations undertake regular prioritisation reviews, align initiatives and agree standards. By taking such an approach, organisations can ensure that they have the best chance of cutting big data down to size and delivering the very real benefits on offer.

 James Riley is global head of innovation at HCL EAS

Read more on Data warehousing