Enterprise data warehouses replace data marts

Businesses are replacing departmental databases - known as data marts - with enterprise data warehouses designed to pool information across the company.

Businesses are replacing departmental databases - known as data marts - with enterprise data warehouses designed to pool information across the company.

"Data marts are very expensive," said Gartner analyst Donald Feinberg. "If you have six departments each with its own data mart, you end up with six hardware systems and six database licences. You also need people who can maintain each data mart."

Businesses often find they end up with the same information replicated in each data mart, he said. They no longer have a single master copy of the data and have to spend more on storage.

"In an enterprise data warehouse, data quality is higher," said Feinberg. "A data warehouse project does not have to take a 'big bang' approach. You can start on a small project and design it around a data warehouse."

Over time, as more projects are added, the data warehouse grows into an enterprise data warehouse, holding all business data.

Auction website eBay runs one of the world's largest data warehouses, gathering 40 Tbytes of data a day, and has 5,000 business users. Speaking at the Teradata European user conference in Lisbon this week, Oliver Ratzesberger, eBay's senior director, architecture and operations, said that like many companies, eBay had too many data marts. "Data marts will kill your business strategy," he said.

Two years ago, Ratzesberger began a project to encourage business units to abandon their individual data marts and use eBay's central Teradata enterprise data warehouse.

"We made this service free so business users could no longer argue they could do data analysis cheaper in a MySQL or SQL Server database," he said. "We told the business: we can provide you with analytics faster than if you bought a server from Dell and implemented the data mart yourself."

Ratzesberger's strategy is to provide data warehouse software within eBay as a central service, run by the IT department, that allows business users to experiment. "We want the business to fail as fast as possible, try out an idea in days, and if that doesn't work, try a new idea and keep trying," he said.

Ebay allows business units to test theories by running data analysis using their own data within Teradata, said Ratzesberger. The idea is that the data warehouse service is not designed for a production system.

"There are no SLAs and users have a six-month limit on the data they upload into Teradata," he added.

If the prototype analysis within Teradata proves fruitful, Ratzesberger said business users would be able to use the prototype as a starting point to develop a production system to run permanently in Teradata.

"Usually our developers would spend 90% of their time fixing data quality issues rather than building the data warehouse. With this new service, they benefit because the prototype identifies data quality issues."

Such issues can be fixed before developers embark on building a production system, he added. "This speeds up development of the software."

Although the data warehouse installation is extremely large, Ratzesberger's approach illustrates how IT directors can reduce the proliferation of data marts in their businesses by giving users a low-cost alternative - an enterprise data warehouse.

Read more on Networking hardware