Heavy data users will pay a huge price for putting all their data into main memory, a business analytics expert said at the weekend.
Speaking at the launch of the 6680 - Teradata's new storage server that combines solid state disc (SSD) drives with rotating memory in the same box - chief development officer Scott Gnau said almost half the traffic in a data system is caused by less than 2% of the data.
Teradata analysis of scores of companies' data flows found "hot" data is about 1.5% of the data, but is responsible for 47% of input/output (I/O) traffic in the system. Some 15% of the data is responsible for 85% of the I/O, and 30% of the data accounts for 94% of the I/O.
Teradata showed two-thirds of corporate data is "cold" or used infrequently. This is data that is typically two to five years old.
Cost-effective data management
Main memory is still the most expensive component of a processing unit, said Teradata president for Europe, Middle East and Africa Hermann Wimmer, so it does not make economic sense to store cold data in main memory.
However, every company has its own pattern of data usage, which is why Teradata has brought out the 6680. The server uses Teradata's existing Active Data Warehouse data management software, which migrates data automatically to the most cost-effective medium, depending on how often it is accessed. This makes it easier for firms to optimise their data warehousing and data analysis.
At the Barcelona launch Teradata also unveiled the 6650, an HDD-based data warehouse server that is upgradable to SSD. The 6680 and 6650 offer cuts of 75% and 25% respectively on floor space and energy use compared with equivalent capacity on earlier models, the company said.
Gnau said sold state discs offer a 20x performance improvement over hard disc drives (HDD). Overall system performance improves about four-fold as the management software learns which data was accessed most often and moves it automatically to the cost-effective memory.
Cloud to remain private
Asked if Teradata planned to launch a "cloud" data warehouse, Wimmer said most companies regard the data used in their data warehouse applications as very sensitive and strategic to their future. This makes it unlikely they would want to have it exposed to a public forum, despite claims of security.
Gnau added that the present cloud business model is for firms to improve their server utilisation from 10% or 20% to over 80%. This gives third party service providers enough margin to eke a living, he said.
"Teradata is already a cloud application, but it is a private cloud," he said. Utilisation rates are above 80%, leaving no margin for third party vendors.