Sergey Nivens - Fotolia

Why it pays to get messy with big data

Big data does not have to be perfect to give useful insights. The ability to make messy data usable quickly could become business critical

Big data is no longer a novelty in corporate circles. It is a regular agenda item at boardroom meetings across the business landscape, and c-level executives are increasingly seeing the value of being at the helm of an insight-driven organisation, using data analytics and information to understand the market in which they operate.

In short, big data is the new normal for organisations that want to understand their customers.

However, there is a misconception among businesses that data analytics can be achieved only through precise, accurate data collection. On the contrary, being exact is not necessarily essential to achieving the desired results.

Sometimes the most comprehensive insights can be created by analysing heterogeneous data from many different sources. In today’s big data world, an organisation that wants to determine a general trend or (dis)prove a broad hypothesis can do so with a ‘quantity over quality’ approach.

As datasets continue to grow and open up, businesses are increasingly having to cope with information that is presented in different ways from a variety of sources. The data can be raw, plain and unstructured, but such chaos could be hiding a gem of insight for the business. The skill is to turn ‘messy’ data into usable, smart data.

Messy data can cause headaches for statisticians or analysts who value precision and process, not least because of the way it is presented. Too many anomalies and errors can lead to useless results.

Common sense is a crucial skill to successfully evaluate the uncertainty of the conclusion and the validity of the quantitative method. By acknowledging that the input data is imperfect, organisations will have made the first step to turning the data into understandable, useful insight.

Accept errors and imperfections

Fundamentally, the way you handle your data determines how much value is yielded from it. With this in mind, imperfections and errors must be an accepted part of the analysis – otherwise the numbers will not be crunched in an efficient and timely manner.

The few erroneous figures will simply be dwarfed by the sheer volume of information, and will not be enough to skew the results when analysed.

Equally, treating big data as something imperfect and imprecise can enable organisations to make superior forecasts and accurate predictions about their particular industry. In fact, the acceptance of messy data makes it easier to eradicate bias and preconceptions about the data being input.

How to make order out of chaos

And it is now easier than ever to make sense of the mess. Cognitive analytics offers a way to bridge the gap between big data and the reality of practical decision-making.

Through cognitive analytics, and deep learning in particular, it is possible for algorithms to abstract information from messy sources, deriving high-level concepts that can be manipulated more easily.

Businesses can capitalise on these recent advances to process and analyse complex data types, such as images, language files and video.

By having a positive approach to tackling data, businesses will be able to make practical decisions and gain a competitive edge in the market.


Harvey Lewis is analytics research director at Deloitte

This was last published in November 2015

CW+

Features

Enjoy the benefits of CW+ membership, learn more and join.

Read more on Business intelligence and analytics

Join the conversation

1 comment

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

Good advice from Harvey Lewis. The data blind spots among the vast troves of data collected - especially in user experience data, comprise just such a critical issue. By capturing the user experience data that only very recently is becoming available AT SCALE, the organizations sees in real time anomalies, i.e., slowdowns, CPU starvation and other events, across the entire organization, that are otherwise missed. The revelations from this data are proving so significant that these blind spots are our obsessive focus at Logfiller Inc.
Cancel

-ADS BY GOOGLE

SearchCIO

SearchSecurity

SearchNetworking

SearchDataCenter

SearchDataManagement

Close