Warren Buffet, chairman and CEO of Berkshire Hathaway says that he does not have an email ID. Surely he does not have to worry about data management. There probably exists an army of professionals to facilitate communication and information management for this legendary investor. But that’s an option, we, as general IT users, do not have. We create data through blogging, micro-blogging, and social media. It is estimated that by 2020, the human world would have crossed 35 zettabytes (ten raised to the power of 21) of data created.
Now, the real conundrum is using this data in your enterprise data warehouse to make business decisions. These decisions can only be successful if you trust your data and have access to it at the right time. Today, sadly, only one out of three decision makers are actually basing their analytical decisions on accurate data.
How to leverage data
Leveraging data is not an option anymore but a necessity. How does, then, one become data driven? The following are some simple ways you can make data your best friend:
1. Use data to decide, not to describe: It is necessary to understand the current situation at hand, but you also need to move further from understanding the problem to exploring solution options.
2. Accelerate your analytics: Let analytics be an engrained system in your organization. Try to make business intelligence (BI) penetrate through the processes and become operational.
3. Do not place restrictions on data: You can never tell where your next big ‘Eureka!’ moment will come from. Don’t disregard unstructured data just because it is troublesome. Now with scores of big data technologies like Hadoop and MapReduce you can leverage even the unstructured data for decisions.
Combining the tools and assets
Do not have a couped up mindset that big data will only affect certain industries. In reality, it can affect you as well. How big is your big data may vary, nonetheless, you need to be prepare your enterprise data warehouse to take on this challenge. Traditional enterprise data warehouses may not have allow for analyzing unstructured data. So forget processing, cleansing and modelling unstructured data. The new perspective is to gather data and leverage it all the time.
Logical data management
Having analytics ingrained into your enterprise data warehouse is costly, complex to manage, and may need people to work on it. You can work through these issues with a logical data management process in place. Here are a few hints:
1. All the data does not have to be in one place.
2. You can consolidate the data and distribute the workload.
3. Give a boost to your analytics project and reduce time needed to deliver value.
4. Manage data end to end.
5. Let the users explore and play with the data.
6. Create data marts on the fly for ad hoc queries.
7. You may work on building a queryable archive.
Lastly, remember distributed data marts are costly to build and maintain. Attain a single version of the truth by reducing data mart sprawl. Keep an eye on the future, plan for a master data management project to achieve the data nirvana.
About the author: John Gillespie, VP and GM, Asia Pacific, Netezza, an IBM Company.
(This article was compiled by Sharon D'Souza from a presentation at the IBM Forum Feb '12)