David Fearne is technical director at Arrow ECS
Whether as consumers or corporate users, we are producing huge amounts of information. It’s not unusual to find 100 gigabytes of storage in a smartphone or consumer cloud offerings starting at 1 terabyte.
First coined in 2001 by analyst Gartner, the term 'big data' describes the fast growing volume, velocity and variety of information. Growth is being driven by an increase in connected devices, such as smartphones and tablets, which will see data produced 44 times faster by 2020, compared with 2009.
With these factors taken into account, there is no doubt that big data is a wide-reaching business opportunity. Many enterprises are anticipating the ability to access multi-petabytes of data in the coming years. But big-data isn’t only about the volume. Some opportunities now occur because organisations are able to analyse fast pacing data in real-time!
The big data challenge
These surging amounts of information present both an opportunity and a challenge. Data can now be stored and analysed, providing insights on anything from weather to shopping habits. However, turning large amounts of data into valuable business insights requires a transformation of not only systems, but also strategies.
So what does this mean for the IT channel when trying to help customers to define a suitable big data strategy?
Deciphering business insights
An optimal method of realising big data's value to a business is by using workshops to demonstrate the area's advantages. When planning a workshop with the end customer, it’s important to first decipher the business insights that they are trying to deliver. This is a fundamental question that should be asked every time a firm engages in a big data opportunity, even within the same customer, as it defines the entire solution.
Valuable insights can vary from business to business, or even departments within one company. For example, a retailer might be trying to predict the future buying habits of customers to better stock the shelves, or it could be looking retrospectively to gauge the success of marketing activity. Some retailers are even looking to develop a one to one marketing channel providing the right promotion, at the right time, when the customer is in the neighborhood of its stores.
Once the key objectives have been defined, it can then be decided how information will be represented. Does the boardroom want graphic visualisations of the ‘big data’? Will they be consuming the results on a desktop or via a mobile application? Or will the information be used to empower more people, so they can take action - or actions taken automatically to streamline processes.
Business needs and big data
A second workshop can be done inside the partner organisation to put the businesses requirements into the context of big data. This should define the type of information that will be used, which could be anything from sales results, sensor information, Excel spreadsheets, existing customer databases, through to the weather or Twitter feeds.
From this point they can decide what the query should look like. Is it better to take a data set and show it against time or enrich that data with other sources to increase its context and unlock previously unseen insights?
A technical solutions workshop will help to scope the actual solution's nuts and bolts.
Again, it's important to consider how to best deliver the end result and how to present it on all necessary platforms, such as tablets, devices and on the web.
Depending on the business requirements, it could be worth investing in on-premise solutions to store and query large sets of data constantly - or process data in movement, so geo-graphic based advertisement becomes reality. Alternatively, it might make sense to use analytics as a service by taking advantage of web solutions for running ad hoc queries. This tactic can also be used as a low cost entry way to prove the value of analytics to an undecided customer.
A technical solutions workshop will help to scope the actual solution's nuts and bolts. This includes the process under which information is integrated, ingested and loaded, as well as identifying what data is suitable for which purpose.
It’s critical that firms ensure they use just the right amount of data. Too little and insights may be lost - but too much could over-complicate the result and cause false positives.
Organisations must also consider how to handle data before storing it. To reference the data scientists at CERN - is it better to store it all and look for a needle in the haystack or burn the hay leaving only the needle?
Big data platforms
Another important consideration is the correct infrastructure. The needs for compute, storage and networking of a solution that is doing real-time analytics versus retrospective are very different. Also important are the types of data being stored, whether structured, semi-structured or unstructured.
Additionally, firms need to decide which analytics platform is most suitable. This once again leans heavily on the types of data being processed and will differ for each. For example, when presented with one structured and one semi-structured source, organisations can use multiple analytics’ engines that perform best on that specific data type.
These vast amounts of information are incredibly useful to businesses, but only when used in the right way. By taking these factors into account, the channel can help businesses cut through the big data hype.
Firms must be taught how to recognise their objectives in order to achieve real value from big data insights. Only then is it possible to realise the area's benefits at all levels of the supply chain.