The main appeal of cloud storage is that an outside entity can take up the slack when data growth occurs and, therefore, IT organisations can avoid capital expenditure on in-house storage infrastructure.
While concerns around security, ease of management, compliance and provider lock-in have inhibited widespread adoption by larger enterprises, cloud storage has become more affordable and prevalent and is a potentially viable option for SMBs and for some use cases in larger organisations.
By moving storage completely into cloud storage, IT shops should be able to cut expenditures in areas such as staffing and training, power and cooling, and resilience capability. But, despite these potential advantages, in the real world few enterprises have gone entirely down this route, as we shall show.
Let’s examine in theory and practice which types of data -- primary, backup or archive -- are best suited to cloud storage.
Cloud for primary data
Locating primary data in the cloud is the most problematic of the three applications. The key aim for most storage administrators is to deliver the best performance possible for production data within the cost and energy budgets available to them. To do this, they deploy the fastest-possible drives and network links to maximise throughput, while siting storage systems close to the servers to minimise latency.
External cloud-based storage cannot deliver the required levels of performance for all types of data. For structured data-type scenarios, it’s just not going to have the performance and for unstructured data users will be sensitive to poor response times that may affect their productivity unless a hybrid cloud solution is deployed that supplements cloud storage with local retention.
So, at the moment, using the cloud for primary data probably means being restricted to unstructured data, and even then there will need to be some local cache. It seems unlikely this situation will change in the near future, as the cost of a wide area link capable of near-LAN speeds remains uneconomic for most companies.
Online backup, where user response times are not a factor, is a better candidate for the use of storage in the cloud and, as a result, a wide range of service providers offer it. Among the best-known is Zmanda Cloud Backup, which offers open source software combined with Amazon's Simple Storage Service (S3) at the back end. The software consists of agents that run on clients and back up either to servers in the client's data centre or to Amazon S3. According to Amanda, the company that provides Zmanda, Amazon S3's fee structure means the cost of online storage is similar to tape but Amazon S3 offers faster recovery and better reliability.
Symantec also offers a cloud backup service by teaming up with Nirvanix, a cloud storage provider. Using a similar approach to Zmanda, the company's Backup Exec Cloud Storage for Nirvanix, which is a plug-in for Backup Exec, allows backup directly to the cloud.
Backup is a natural application for the cloud. Although it requires the transfer of large amounts of data initially, incremental backups thereafter will be relatively low in volume. And because by nature the backup data will be remotely stored, it acts as a form of protection against local disasters.
The downside to be aware of with cloud backup is the time to carry out restores. Because these tend to be large in volume they are often carried out offline with backup sets sent to customers via physical media. So, be aware the speeds you gain for uploads of backup will not usually be matched by the speeds for a restore.
Backup involves a little data and follows a frequent schedule, while archiving involves larger chunks of data and happens more occasionally. The value of the data differs too. Backup data is current and therefore very valuable in day-to-day business terms, while archived data is by definition older and will not change. This means archived data must be securely stored, but restore times are not as critical, and integrity is more important than performance.
An example of a backup vendor that offers cloud archiving is CommVault, whose Simpana software provides tiered archiving using cloud-based storage provided by Farline. It allows you to archive stale data directly from the data centre and from remote sites to cloud-based storage, providing a single repository for archives and as the basis for a disaster recovery plan.
The advantage of cloud archiving is that, in addition to the lack of hardware and associated costs that all cloud implementations offer, there’s greater reliability because it uses disk as its medium. Tapes, still predominant for in-house archiving, may be less reliable in the long term if not properly managed. For example, organisations should periodically migrate archived data to new tapes to avoid problems such as tape stretching, evaporation of lubricants and to avoid software and hardware incompatibilities that creep in over long periods of time as a result of technology updates.
Hillingdon puts primary data in the cloud
Hillingdon Council has plans to put some of its primary data into the cloud and expects the project to go live by the end of 2011. By the end of 2013, according to Roger Bearpark, head of the council's IT, 10 TB -- or about 40% of the council's primary data -- will be located in the commercial cloud. "We did the first trials this month," Bearpark said. "It went very well. It was desperately dull, which was just what I wanted."
The council plans to make available documents that are open to public access via the cloud. Bearpark described them as low risk, typically items such as "rubbish collection dates and others with no security implications, but not child adoption records."
The advantages of the move include flexibility of access to the data. "We can provision it in-house now, but it's complex and expensive, while in the cloud it's accessible via an Internet browser. You just need strong authentication and you're in," Bearpark said. "Another advantage is the flexibility to share data with members of the public and partners at little or no cost."
Bearpark said Hillingdon has partnered with a cloud provider that is "a major international brand" but declined to be more specific. "We talked to three different providers. This was the best commercial deal as all the services on offer were largely indistinguishable."
"Our aspiration is 100 percent uptime, and the incentive for the cloud partner to achieve that is to avoid reputational damage. We didn't delve into the detail of how they would achieve that, but it was a key requirement [in negotiations]," Bearpark said.
For the future, Bearpark said Hillingdon is not considering either cloud backup or cloud archiving.
Fujifilm Diosynth Biotechnologies backs up to the cloud
Fujifilm Diosynth Biotechnologies, which offers process development and drug manufacturing facilities and services from its Teesside facility in the UK, uses London-based Frontier Technology’s cloud services for backup and disaster recovery purposes.
The company chose this route because, according to Campbell Kay, IT manager, it was looking for a solution that was simple to implement without too much in upfront costs. Kay said, “I wanted real-time replication for our business-critical data, but as the company doesn't have a second site in place already, this was an expensive option for us to put in place ourselves. Using a cloud services provider, therefore, seemed to be an obvious choice."
The company currently backs up 4 TB to 5 TB but also wants the ability to expand as data volumes grow.
For Kay, the choice of provider was all about "taking the worry out of the situation as much as possible. This meant looking at what would happen in the event of a disaster or server outage, and that our data would be protected. I expected any provider that we looked at to solve those problems first and foremost,” he said.
Because of the markets the firm works in, it has to meet stringent industry regulations and client requirements for business continuity.
"The partner we work with takes care of everything,” said Kay, “and I know that if there is a problem on their end, everything is backed up to another location as well. It’s like having two DR plans in place.”
Fujifilm Diosynth Biotechnologies assessed alternatives such as non-cloud solutions, but Kay said most did not offer the level of protection he was after. “Some of them were based on taking snapshots of the systems or scheduled backups,” he said. “The problem with these options was that they still allowed for significant amounts of data to be lost. After doing a bit of research, I came across the term ‘real-time replication’ and realised this was what the company needed to reach not only our recovery time objectives but also our recovery point objectives."
Cost was also a consideration, according to Kay, who said implementing a cloud computing solution meant upfront costs were a third cheaper than implementing a non-cloud alternative.
The system--which is based on Vision Solutions' Double-Take--works by copying changes automatically to Frontier's data centre. Because the changes are relatively small, the process has low bandwidth requirements, with latency usually about 10 to 15 milliseconds over the IPSec VPN connection.
"All our data is then held on the Frontier SAN in a Tier 4 data centre and is backed up daily in accordance with the SLA that we have in place," he said. "System restores can be completed within 30 minutes of the invocation to switch over to the cloud service provider."
Reassurance about the safety of the company's data is provided by an SLA that guarantees a maximum recovery time of one hour from invocation and that includes one disaster recovery test annually.
"There are two main benefits for the approach that we have taken," Kay said. "The first is that we have a service in place that protects our data assets and our server workloads against the risk of failure. Secondly, the back-end management is taken out of our hands, which leaves us able to concentrate on other business IT requirements."
Manek Dubash is a UK-based freelance journalist.
This article is part of our package on cloud computing trends.
This was first published in June 2011