E-mail is the data type that nobody wants to know about - it is a huge problem. The storage teams do not want to know, the mail teams do not know how to cope, and the compliance issues grow just as fast as the e-mail volumes. Corporate e-mail is a ticking legal and IT timebomb.
Recently I was reading about a major scandal at Bear Sterns allegedly exposed through the discovery of an e-mail message from a fund principal that apparently stated that their fund was going to be "toast".
The first thing I thought about this was that (if true) it was a fantastically stupid communication to put in an e-mail exchange. Second, I wondered why it took so long to find this mail - surely such high-profile financial managers would have their mail exchanges monitored automatically and an exchange like this should have rung every major alarm bell in the firm within seconds. Of course, they could have been using an external system to get around that we do not know at present. But this case once more highlights the limitations of e-mail monitoring and e-discovery, and conversely the value of e-mail archiving.
E-discovery is in many regards simply glorified enterprise search technology, but with the added ability to apply legal holds to data. Just as enterprise search is limited by the quality and location of the content it indexes, so too are e-discovery tools. In the case of e-discovery the limitations are often more severe: evidence may or may not be conveniently located in an e-mail message, as seems to be the case at Bear Stearns.
More commonly evidence has to be culled from not only e-mail stores, but also from instant messaging systems, document systems, enterprise resource planning systems, financial and business applications, external drives, and so on. The idea that e-discovery is limited to mail - as many suppliers (and worryingly many buyers) seem to think - is naive in the extreme. Yet this misplaced belief is based on the reality that the bulk of the data you will have to search will indeed be mail. Mail represents the largest form of data in any organization, typically by an order of magnitude of 10x or more.
But here's the rub. Most of that e-mail mountain consists of redundant data or as the technical terms goes, "rubbish." As we discuss at length in the CMS Watch e-mail Archiving Report, typically 80% of mail data consists of a duplication. Yet any search tool has to treat each piece of data equally, slowing the process down massively and shooting discovery costs through the roof. How much more sensible to use an archiving method to capture, filter, and reduce that volume - and ease the burden and cost of discovery?
We estimate that the cost of 1GB of storage is about 10p however, the cost of legal discovery on 1GB of storage would be at least £1000, so storing everything may seem cheap on the one hand, but is in fact spectacularly expensive should something go wrong. Again, most EAM tools today will reduce the storage volumes by around 80% at the first pass, so it makes a great deal of sense to use such tools, reduce the volume of e-mail you are storing, free up the Exchange servers and generally live a happy life. Yet few firms bother to do this.
From our research it seems that the reason firms do not use such tools is that a: teams that operate the mail systems, those that have legal responsibility over the contents of messages and those that operate the datacentres are separate, with different budgets and different priorities - it is that old business/IT disconnect again. It is a real shame as there are many products on the market these days that can return a very fast return on investment and more importantly tidy up the mess that is your e-mail system quickly. So even if compliance is the key driver when considering buying an e-mail archiving and management (EAM) system, the fact is that vastly improved efficiency and dramatically reduced data volumes comes as part of the parcel.
So, what did we learn from the Bear Stearns scandal? Not much really, other than mail (and messages) continue to be the key "gotcha" elements of the data mountain, and that we need to monitor and manage them ever more closely. Though the monitoring elements are far from mature, EAM tools today archive and filter very efficiently indeed. The need to take mail and mail content seriously is now an imperative, and building a strategy, agreeing methods and policies, and selecting the right tools - however complex - is a must.
Alan Pelz-Sharpe, Principal Analyst at CMS Watch leads a panel on E-mail Management and Archive - How to Spend Wisely in the keynote programme at Storage Expo the UK's definitive event for data storage, information and content management.
The event features a comprehensive free education programme and over 100 exhibitors at the National Hall, Olympia, London from 15-16 October 2008.