alphaspirit - Fotolia
Flash storage provider Pure Storage published an interesting report on 2 December entitled European Big Data – Big Failure.
Pure Storage chose to focus on the fact that UK businesses lose up to £20m per year through missed opportunities because they lack accurate information when they really need it. It also reported that 78% of companies believed they could boost performance by just over a fifth (21%) if they had faster insights.
All of which is interesting but I have to admit my attention was drawn more to the finding that 72% of companies said they collect data that they never use because either data processing is too time consuming, it’s too expensive to process the data or they don’t have the skills internally to do it.
Think about that for a minute. Almost three-quarters of the businesses surveyed had the technology to collect all the data they needed but they never used that data. To get an idea of how absurd that is, imagine if three-quarters of grain farmers had silos to put their grain in but no way to take it out.
But here’s the thing, can anyone seriously imagine that farming as an industry would allow a situation where it created silos for the grain but made it too expensive and time consuming for farmers to do anything with it afterwards?
And yet that’s what’s happened in the IT industry.
Worse still, unlike farmers, businesses have been told they need to harvest everything in all their fields, not just the grain fields. Which means that, unlike the farmers, their silos are filled with all sorts of inedible crap as well as grain. So not only do they have to find a way to get at the grain to use it, they also have to be able to separate it from all the other junk that’s in the silo too. No wonder they think that process is time consuming and expensive.
There’s a tendency on the IT industry’s part to suggest this state of affairs is somehow the customer’s responsibility. But customers wouldn’t be in a near-panic to store all their data if the IT industry didn’t keep telling them that if they didn’t they’d lose out to their competitors or be unable to keep pace with changes in the market.
Losing out to competitors or being unable to adapt to changes in the market is a big fear when it comes to data. As James Petter, VP EMEA, Pure Storage, said in the press release accompanying the report: “As companies gather more and more granular data on what they do, the potential to gain understanding and plan accordingly is not just a profitable undertaking, it is a necessity. Transformation is being forced on organisations at an ever-increasing pace. They must adapt to new ways of doing business, new markets and new practices – or die.”
“Die” is a pretty drastic word in this context. In any case, in an ideal world, organisations wouldn’t be forced to gather more and more data that they don’t have the time to scrutinise but would concentrate instead on applying filters to separate the junk from the good stuff before they stored it.
Unfortunately, the current model leaves them trapped in a Catch-22 situation where the process of accessing data becomes more complicated and expensive as they store more and more data (and more and more of that data is irrelevant or worthless).
Something has to give. Otherwise a lot of companies could “die”, starved of the data they might need to improve their business even though their silos are filled to bursting.