Avoiding the potholes on the way to cost effective MDM

This is a guest blog by Mark Balkenende, a manager atTalend


Master data management is one of those practices that everyone in business applauds. But anyone who has been exposed to the process realises that MDM often comes with a price. Too often what began as a seemingly well thought out and adequately funded project begins accumulating unexpected costs and missing important milestones.

First we need to know what we’re talking about. One of the best definitions of a MDM project I’ve heard is from Jim Walker, a former Talend director of Global Marketing and the man responsible for our MDM Enterprise Edition launch. Jim describes MDM as, “The practice of cleansing, rationalising and integrating data across systems into a ‘system of record’ for core business activities.”

I have personally observed many MDM projects going off the rails while working with other organisations. Some of the challenges are vendor driven. For example, customers often face huge initial costs to begin requirements definition and project development. And they can spend millions of upfront dollars on MDM licenses and services – but even before the system is live, upgrades and license renewals add more millions to the programme cost without any value being returned to the customer. Other upfront costs may be incurred when vendors add various tools to the mix. For example, the addition of data quality, data integration and SOA tools can triple or quadruple the price.

Because typically it is so expensive to get an MDM project underway, customer project teams are under extreme pressure to realise as much value as they can as quickly as possible. But they soon realise that the relevant data is either stored in hard-to-access silos or is of poor quality – inaccurate, out of date, and riddled with duplication. This means revised schedules and, once again, higher costs.

Starting with Consolidation

To get around some of these problems, some experts advise starting small using the “MDM Consolidation” method. Because this approach consists of pulling data into an MDM Hub (the system’s repository) and performing cleansing and rationalising, the benefit is that consolidation has little impact on other systems.

While consolidation is also a good way to begin learning critical information about your data, including data quality issues and duplication levels, the downside is that these learnings can trigger several months of refactoring and rebuilding the MDM Hub. This is a highly expensive proposition, involving a team of systems integrators and multiple software vendors.

In order to realise a rapid return on MDM investment, project teams often skip the consolidation phase and go directly to a co-existence type of MDM. This approach includes consolidation and adds synchronisation to external systems to the mix. Typically data creation and maintenance will co-exist in both the MDM system and the various data sources. Unfortunately this solution introduces difficult governance issues regarding data ownership, as well as data integration challenges such as implementing a service-oriented architecture (SOA) or data services.

There are other types of MDM, each with its own set of problems. The upshot is that the company implementing an MDM system winds up buying additional software and undertaking supplementary development and testing, incurring more expense.

An alternative approach

Rather than become entangled in the cost and time crunches described above, you should be looking for vendors that provide a solution that lets you get underway slowly and with a minimum amount of upfront costs.

In fact, part of the solution can include Open Source tools that allow you to build data models, extract data, and conduct match analysis, while building business requirements and the preliminary MDM design. All at a fraction of the resource costs associated with more traditional approaches.

Then, with the preliminary work in place, this alternative solution provides you with the tools needed to scale your users. It is efficient enough to allow you to do the heavy development work necessary to create a production version of your MDM system without breaking the bank.

Once in an operational state, you can scale up or down depending on your changing MDM requirements. And, when the major development phase is over, you can ramp down to a core administrative group, significantly reducing the cost of the application over time.

You should look for vendors offering pricing for this model based on the number of developers – a far more economical and predictable approach when compared to other systems that use a pricing algorithm based on the size of data or the number of nodes involved.

This approach to MDM deployment is particularly effective when combined with other open source tools that form the foundation of a comprehensive big data management solution. These include big data integration, quality, manipulation, and governance and administration.

By following this path to affordable, effective MDM that works within a larger big data management framework, you will have implemented a flexible architecture that grows along with your organisation’s needs.