By Antony Adshead, UK Bureau Chief, SearchStorage.co.UK, and Danny Bradbury, Contributor
Why use email archiving services? The primary reason is because they address a major headache for the IT department. While email is vital to the flow of communication necessary to business nowadays, it's difficult to manage and retain, and also presents technical, business and legal challenges.
Email archiving applications seek to address those problems by porting an organisation's email to a separate archive repository that can be indexed and searched, which is important for compliance regulations. Many email archiving products also incorporate ways of reducing the volume of data that must be stored, for example, by keeping only one instance of an attachment that occurs on many emails, often leaving only a "stub" pointer on the email application to the fully stored email in the archive, thereby making the email store work more efficiently.
E-discovery is a key driver of email archiving, according to Bob Tarzey, analyst and director at analyst firm Quocirca. "From a discovery point of view, if you're asked a question about what information you hold on an individual, archived emails are one of the main sources you check to find this stuff," he said.
But handling these email archiving chores in-house is a technical and legal challenge. Email is perhaps one of the biggest drains on corporate storage resources, and employees have a propensity to attach large multimedia files. When IT departments try to control the size of user mailboxes with quotas, employees save email database files -- such as Microsoft PST archives -- locally, which results in emails that are not centrally backed up or searchable. Both of these practices are bad from a legal and business viewpoint.
Email archiving services
These kinds of challenges have caused many organisations to adopt email archiving services to offload the burden of storing archived data from the corporate data centre and to shift the chore of readily responding to e-discovery requests to an outside party. For small- and medium-sized enterprises (SMEs) in particular, there's appeal in a service offering. For companies managing fewer than, say, 1,000 mailboxes, it may make sense to outsource email archiving to an email archiving service provider.
While email archiving services have generally been adopted by smaller companies, we are starting to see interest from larger firms for good financial reasons, according to Brian Hill, a senior analyst at Forrester Research.
"We see that organisations continue to adopt SaaS-based solutions at a good clip, for reasons of lower TCO and rapid deployment times," he said. "But there are other factors, too. For example, Software as a Service-based [SaaS-based] models hit operational budgets rather than capital expenditure budgets, which is important for many companies."
Hill predicts a likely surge of growth in the email archiving services market during the next year or two. In the European market, Gartner predicts an average annual growth rate of 16.3% to 2013.
How do email archiving services work?
Email archiving products comprise two main functions -- a means of capturing email messages and effecting their movement to a physical archive, and an archiving engine that indexes email content for search and applies policies that dictate their subsequent movement and retention.
The same things happen in an email archiving service, but offerings vary in the way those functions are split between the customer site and the provider. Some services work almost entirely in the cloud with email traffic passing through them before reaching the customer; with other services, the customer's email infrastructure receives the email traffic, which is then journaled or copied to the service provider's data centre.
Some providers simply offer cloud-only or journaled-only email archiving services, while others offer both.
Email archiving services also vary in their sophistication. You'll pay more for advanced analytics, management capabilities, better security, greater scalability and the ability to handle multiple data types. Google's Message Discovery cloud service, for example, offers a year's worth of retention for $25 per user; retention for 10 years costs $45 per user. Meanwhile, Rackspace offers webmail mailboxes at $1 per month, hosted Exchange mailboxes at $12 per month and email archiving at $3 per user per month.
The email journaling method
Journaling email data involves taking a copy of the message, along with metadata such as the send date and a record of those involved in the conversation. This data is then stored separately so that metadata and keywords can be used to index the email.
In a journaled environment, companies that already have a considerable investment in an email infrastructure can continue to manage their email locally. Their existing email servers store mail and are used for operational purposes. At the same time, however, journaled data can be used by the email archive service to facilitate archiving services.
The journaled model is implemented in a number of ways. For example, Smarsh provides a journaling appliance designed to be installed at the customer's premises. The appliance indexes journaled mail and copies it to a WORM drive so it becomes tamper-proof, then replicates journaled email to a remote data centre. One of the benefits of this model is that it gives administrators a second point of failure. If their own email servers fail, the appliance can queue incoming email until they are ready to receive it again.
LiveOffice offers two options. Customers can journal mail directly to LiveOffice's servers or they can set their MX records (the part of their organisation's domain name system entry pertaining to email servers) to LiveOffice's servers. This second option routes all of an organisation's incoming and outgoing email through the LiveOffice infrastructure, enabling the company to journal mail en route.
Mimecast, on which Iron Mountain also bases its cloud-based email archiving solution, also works this way.
The other option is to have operational data stored in the cloud as part of a managed cloud email service with journaled data then sent to the local implementation. This is the model adopted by Symantec in partnership with Microsoft Exchange Hosted Services.
How to select an email archiving service
When choosing an email archiving service, administrators will want to consider other factors than deployment models.
Some companies, for example, will find the ability for users to access archived email directly from the desktop important for productivity purposes. Mimecast is one company that offers an Outlook plug-in so users can search up to 10 years of archived email without leaving their email software.
Another issue to consider is social media and instant messaging (IM) integration. For many companies, email is one of several channels their employees use to communicate. Autonomy comes from a background rich in content analysis and offers IM integration, indexing of video and audio, and can even store and index telephone conversations. Meanwhile, Smarsh appears to be focusing on broader social media; it offers the ability to manage and archive activity on services such as Twitter and other social media platforms.
Some of these email archiving services can also be used as a platform for specialist services. For example, CaseCentral, another Symantec partner, focuses on e-discovery for legal purposes. It provides services such as near-duplication analysis, which groups together documents that are almost (but not quite) the same from within the email archive so that litigators can better analyse them. It also provides concept identification services that enable analysts to trawl emails and identify them based on the concepts raised in the text.
Representative email archiving services suppliers (not an exhaustive list)
Autonomy: A market leader in content discovery and management that offers on-site, hybrid and email archiving services.
Google Message Discovery: Operates totally in the cloud with a Web interface.
Iron Mountain/Mimecast: Customer retains own email infrastructure and archives to Mimecast data centre.
LiveOffice: Completely in the cloud, apart from software on your email server journaling email traffic to a LiveOffice data centre.
MessageLabs/Symantec: Can be either a purely hosted setup or used as an on-site archiving device.
Microsoft Exchange Hosted Archive: Sits between your Exchange infrastructure and the public Internet, where it manages, archives and stores email traffic.
Proofpoint: Hybrid with on-site appliance and off-site archive.
Rackspace: The company's Rackspace Archiving services copy all message data as it's created to two separate storage services -- Rackspace's own Cloud Files and Amazon's Simple Storage Service (S3).
This was first published in July 2010