Business Link for London, the government support service for SMEs, has spent the last two years reducing duplicate data in its database from 25% to 2.5%.
The organisation said this has boosted efficiency and estimated a saving of over £100,000 in that period by avoiding duplicate mailings to companies on its mailing list or to non-existent company addresses,
Business Link for London is the product of a 2001 merger of nine organisations. When it was formed, individual databases from each group were merged into a single repository to enable end-users to see the same information they had accessed when they worked for independent organisations.
Mike Pratt, data integrity manager at Business Link, faced the challenge that while everyone could see their data, there were multiple records for many companies. Problems were compounded by the decision in 2004 to put all the data into a brand new Peoplesoft CRM system.
Most organisations merge or delete records when faced with duplicate data, but Pratt said this strategy was not practical for Business Link.
“A common problem is that people delete records and realise they made a mistake,” said Pratt. “And it is difficult to put the data back.”
Business Link, he said, “dumped data into the CRM system without regard to duplication.” It then used the Firstlogic de-duplication tool from Business Objects to identify that 25% of the data in the Peoplesoft CRM was duplicate records.
Pratt used the tool to create a “master-subordinate” process, where a user identifies one main record in a duplicate set as the master, with all other duplicates deemed subordinate.
The benefit of this approach is that the subordinate records are filtered out but can still be accessed if necessary.
Through this data cleansing strategy, Pratt hopes to bring down the rate of duplicate records to 0.5%, as users identify new master records from duplicate sets.