Modern organisations are thick with data, but where is the best data these enterprises have to offer? Where is the most appropriate, the most up-to-date and the most accurate data? How do we create a representation of the business world that users find recognisable?
John Morris, author of the BCS book Practical Data Migration, was prompted to ask questions like these when he first got involved with data migration 10 years ago. In his book he has tried to offer techniques and strategies for ensuring that data migration projects achieve maximum return on investment.
Talking from past experiences he said, “We had gone through the painful process of building a migration tool, stocking it with extracts taken from the most legitimate source available (a rather cranky old database created for regulatory purposes), only to find it all rejected out of hand by our user population.”
It turned out that each set of users had, out of exasperation or necessity, created a separate store of the data items they needed to do their work. Of course, each store was built to a different format and standard. Some were in Excel, some were bastardised versions of a data-gathering Access database that had been left behind from a previous migration exercise. Some were in non-standard database formats using tools officially superseded a couple of desktop versions ago.
To the technologists these stores represented the worst possible nightmare of design and build aberration. To the users they were the best fit to their needs and, being closer to the coalface of day-to-day activity, they were also the best matches to external reality.
“We were faced with the classic migration dilemma: stick to the official policy and use the discredited corporate data store, or listen to our users and use the best of what they had to offer,” said Morris.
The latter course was chosen, entailing considerable additional effort. Each data store had to be examined for quality and rejigged to fit the new data structure. Extracts had to be hand-coded or even re-keyed. However, once the project was allowed be led by the business it became a shared problem.
The question now became, “How are we (the business) going to preserve the best of our data and add value to it by enhancing its quality, as we load it into the new tool?”
From this Morris learned the most important rule of successful data migrations: to guarantee success, data migration has to become a business problem, not a technical one.
For information on BCS membership go to www.bcs.org/membership
Vote for your IT greats
Who have been the most influential people in IT in the past 40 years? The greatest organisations? The best hardware and software technologies? As part of Computer Weekly’s 40th anniversary celebrations, we are asking our readers who and what has really made a difference?
Vote now at: www.computerweekly.com/ITgreats