Read any IT publication and it is hard to move more than a couple of pages before the merits of big data are being promoted. Indeed, such features often go hand-in-hand with the idea that every organisation is rushing to exploit the big data bounty to be had. But is this an accurate image of what is actually taking place in the world of everyday business? Freeform Dynamics recently undertook a survey to find out what organisations think about big data.
Perhaps the most interesting point thrown up by the online survey is just how little understanding the marketing efforts around big data have managed to generate (see figure 1).
Our research indicates only a very small percentage of respondents state they have a clear idea of understanding of the advanced technologies now hitting the markets. This should be a matter of concern to suppliers and IT professionals. When coupled with the result that an equally small number have a clear idea of business benefits potentially available through the use of big data solutions, it is clear there is much to do before big data can begin to enjoy widespread adoption.
The scale of the educational challenge faced by suppliers should not be underestimated since online surveys such as this usually attract a disproportionate number of respondents familiar with the area under investigation or those with strong views on the topic.
In this context the far higher numbers of those surveyed willing to say they do not have a clear understanding of either the systems available or the benefits that could be achieved almost certainly indicates that real levels of awareness in the IT community as a whole are at very low levels. Suppliers need to address these fundamental issues head on, rather than continuing to pump good-news marketing messages if most companies are to take advantage of new data analysis tools to generate business value from their expanding hordes of data.
As might be expected when levels of understanding of new data storage technologies are low, the new solutions are currently not very widely deployed (see figure 2).
But all is not doom and gloom. The chart also indicates that even though understanding is low, use of such platforms is expected to expand substantially over the course of the next three years. And this expected expansion is also reflected by the results shown in figure 3.
There is plenty of evidence that IT professionals see that new storage solutions will allow them to be able to tackle problems previously considered too difficult or expensive to address. Perhaps even more importantly, it is also apparent many respondents believe these advances will provide the opportunity to tackle key business requirements head on.
These results are significant. We know many organisations report difficulty extracting value from their data sources, both structured and unstructured, despite many years grappling with business analytics and intelligence analytics.
The major marketing drives around big data may offer organisations an opportunity to enhance their business intelligence (BI) capabilities which have often suffered from underinvestment.
With analytics nudging towards being vaguely fashionable, now is the time to bring forwards good investment initiatives.
Resources on big data
But even if the supplier community can become more educative when talking about BI and analytics, never mind big data, there are still many hurdles to be overcome.
Of these it is the severe dearth of skilled human resources, able to run such tools and make sense of the answers generated, that will be the hardest issue to address.
Indeed, it often appears as if the need for people comfortable with data analysis and capable of asking the right questions is too easy to overlook when most marketing budgets are concerned only with technology.
The role of people in BI is underappreciated. Neither big data nor run-of-the-mill business analytics initiatives will succeed in the long term until organisations recognise the requirement to train staff thoroughly on the tools they use.
In big data, as in almost all areas of IT and business at large, technology alone is never enough.
Tony Lock is programme director of Freeform Dynamics
This was first published in October 2012