There are lots of surveys out there that will tell you how businesses understand and appreciate the value of having good quality data to make decisions against. But what most of these surveys don’t discuss is whether those businesses are putting their money where their mouth is on data quality management. At Bloor Research, we decided to find out – and the fact is that they aren’t.
Here are some of the headlines from this as-yet-unpublished research on the importance of data quality:
- Out of nearly 700 organisations, 83% said they appreciate the business value of good quality data. Despite this, 45% claimed that budget limitations hold them back from making the most of their data as a business asset.
- On average, just 16% of budgets are dedicated to leveraging data.
- Estimated data growth has been slightly under 100% over the past five years, and the average amount of data within organisations is expected to more than double over the next five years.
- 75% of respondents recognise that data growth will lead to future challenges, and 70% believe that failure to successfully manage the growth will ultimately limit their organisation’s competitiveness. Ominously, almost 45% said they think their organisation will struggle to meet data-growth demands.
There are some noteworthy points buried in these bald statistics. For example, the 83% cited above agreed or strongly agreed with this statement: “My enterprise appreciates the value of accurate business data.” However, it’s not as simple as that. Since this was a global survey, we were able to look at the results by geographic region – and while all regions were approximately equal in terms of their overall totals, in North America and the Asia-Pacific countries in particular the “strongly agree” camp outweighed those that merely agreed. In Europe, on the other hand, simple agreement was nearly twice as popular as strong agreement. Why is this?
Walking the walk on valuing good quality data?
Well, you could argue that it is about perception. That is, we were asking what respondents thought of their companies, and perhaps Europeans as a whole are more sceptical about employers than their American and Asian counterparts are. Continuing in that vein, we found that IT people were generally more sceptical (with more agreeing than strongly agreeing) than respondents from the business side. Or maybe it’s that the business talks the talk on valuing high-quality data but IT doesn’t see their companies walking the walk. Certainly, this is backed up the nearly half of our respondents who claimed that budgets were preventing them from fully exploiting their data (though perhaps we should have also asked about the effect that the recession has had, as that might help explain why that figure is so high.)
Of course, part of the budget issue is that by an overwhelming margin, the largest part of the typical IT budgets is assigned simply to keeping the lights on and the systems running. Hence the relatively low figure of 16% that is allocated to getting the most value out of corporate data. There were some geographical differences here, too: the average was 14% in Europe, 17% in North America and 25% in Asia-Pacific. Within Europe, the UK and Ireland had the highest portions of budgets dedicated to leveraging data, equivalent to the overall average of 16%, but in some countries the level was as low as 9%.
If it’s true that information is a company’s greatest asset (and I certainly think it is), then the budget numbers suggest that organisations in the Asia-Pacific region are moving forward faster than the rest of the world. The same applies to data growth: Asia-Pacific companies reported larger increases in data volumes over the past five years than businesses elsewhere and expect greater growth in the next five years – no doubt a reflection of their booming economies. Interestingly, North American respondents reported the lowest levels of data growth and foresee the least growth in the future.
What is also worrying about these statistics is the 25% of people who don’t think that data growth will lead to future challenges and the 30% who appear to be completely sanguine about how failing to handle increasing amounts of data could affect their competitive position. The potential problems go beyond maintaining good quality data – I could write a whole article just about the challenges of storing and processing more data, especially when there are increasing requirements to load, query and view data in increasingly complex ways and with increasingly rapid response times.
New top priority: putting good quality data to effective use
Then there are the nearly 45% who don’t think their companies are going to be able to cope with the coming data growth. There are some further parallels with the other results here: respondents are more bullish about managing growth in Asia-Pacific and the most pessimistic in North America. And again, IT people tend be more pessimistic than their business counterparts.
All of this is interesting, but it is only facts and figures: it doesn’t tell us why there is a mismatch between the perception of how important data is and the actual practice of leveraging it. I think we’re in the middle of a paradigm shift. For the past 20-odd years, we all have been focused on applications, but now the emphasis is shifting towards information and making effective use of high-quality data – rightly so, but the message hasn’t filtered all the way through the organisation yet.
It can be seen in the increased interest in pervasive and pre-emptive data quality and in the take-up of master data management, data warehousing, data governance and operational business intelligence, just as examples. It’s a bit like turning an oil tanker: it’s only a long time after you turn the wheel that you start to see the ship turn. My guess is that if we repeat this research on the value of data and the importance of data quality a couple years from now, we will get significantly different results.
Philip Howard is a research director focused on data management for Bloor Research. He tracks technologies and processes such as databases, data integration, data quality and master data management. Howard has worked as a Bloor analyst since 1992; he also writes frequently for IT publications and websites and is a regular speaker at conferences and other industry events.