Reading through the vast amount of analyst comment from Creative Intellect Consulting, Gartner, IDC, et al, it's hard to see how CIOs get any sleep. Over the last year, however, there appear to be two key themes that are at the top of the CIO agenda, Cloud and Business Intelligence. Now Teradata has thrown a spanner into the works by saying that BI is NOT a Cloud application.
Teradata's view, based on their customers who are almost exclusively in the top 3000 companies, is that BI and data warehousing are hugely IO intensive applications and the problem with Cloud based solutions is that they are more focused on the provision of compute resources such as virtual machines than they are IO. In addition, as Cloud operators grab any spare hardware when you need to scale, and high performance BI requires parallel systems, you would be seriously limited by the slowest machine in their environment.
Another problem for the current generation of Cloud service providers is that their money comes in the increased utilisation of hardware. According to Scott Gnau, Chief Development Officer, Teradata," our systems already run at 90-95% utilisation making them deploying as a private Cloud today so there is nowhere for the Cloud provider to make any money."
However, Teradata did concede that for smaller companies, particularly the SME who lack the technical expertise and resources, the use of Cloud based BI services may be a solution. They were even prepared to concede, under questioning, that it is possible that some form of Cloud infrastructure capable of doing high performance BI might eventually exist, but not in the short term.
What Teradata does believe might happen is a stratification of those Cloud providers who want to offer BI services. For this to happen, would require a significant investment in hardware and a bespoke solution that is also capable of being multi-tenant driven. In this scenario, the provider would be able to give a Service Level Agreement (SLA) for guaranteed IO.
Martin Willcox,Director of Product and Solutions Marketing, EMEA, says that Teradata has been having conversations with some Cloud providers about the idea of a BI Cloud but at the moment there isn't the demand from Teradata customers or really from the Cloud providers for such a service. Teradata's view is clearly at odds with a number of other vendors such as IBM, Microsoft, HP, Oracle and SAP Sybase who are already building out Cloud infrastructures that are capable of supporting their customers BI requirements.
Willcox also points out that Teradata has focused on the philosophy of making the DBMS as self managed as possible freeing up the DBA to deal with other, more pressing, issues. Among those are the complexity of removing the data warehouse silos that exist in a lot of companies and building a single, coherent, data warehouse against which they can ask questions. But at a fundamental level, a lot of organisations struggle with consistent data models, data transformation and naming schemes. Willcox believes that the only way to resolve this is internally, outsourcing your data warehouse and BI and then expecting the outsourcer to deal with this is outside of the scope of the outsourcer.
There are other issues that have to be overcome by the whole Cloud industry. These include the security of data both at rest and when moved to end user devices, where the working sets are held and the bandwidth needed to move all that data around. On top of this, the whole issue of application latency has to be dealt with and this is something that is already causing serious problems for application developers.
With many of Teradata's customer base in the financial industry, Willcox says that the idea of moving all their data to a Cloud provider is something that causes major internal and compliance issues. This is not just about live systems but also covers test and development and it is exactly this latter scenario that has driven many organisations to take a long hard look at the Cloud.
To be able to properly test an enterprise application you need real live data and the ability to test against many more users that you actually have. Internal resources are rarely enough so the idea of being able to deploy to the Cloud and just acquire the necessary resources short term is just good sense. In addition, test tools are extremely expensive so with vendors such as HP and IBM making their test tools available through Software as a Service, it is now possible for test and development teams to really stress an application.
However, Willcox still believes that BI is a long way from taking advantage of this. He points to the security problem of putting data onto a Cloud providers site, even if sent on tape and held in a secure area. As to uploading the data for short term tests, Willcox says that there are few companies, if any, that have enough bandwidth to actually do this.
One of the big drives from Teradata has been to get customers and developers to run their applications on the Teradata platform, so that they are as close to the data as possible. With Cloud, where there are security concerns over data, we have seen people move the applications into the Cloud and keep the data locally. Willcox dismisses this as being a viable option for high end BI because the latency inherent in the network is too slow to allow the application to work effectively.
However, this may be about to change, at least for Teradata customers. Willcox says that they have been doing a lot of benchmarking and testing and the Teradata labs in the US and on some customer sites to address the issue of the movement of data, compliance and data security. While there is a lot of maturing needed across all the various parts of the ecosystem, there are improvement happening and Teradata is preparing new software.
Despite all of this, Willcox still maintains that there is still a significant lack of customer demand for Cloud based BI solutions.