As organisations struggle to cope with data volumes that are doubling each year, so it becomes increasingly important that data storage policies are put in place. Before entering into a discussion as to why it is important to have a data storage policy, it is first necessary to establish what one is.
Put simply, a data storage policy is a set of procedures that are implemented to control and manage data within the organisation. These can range from policies that determine how data is collected and stored, to a set of applications that manage all aspects of data. It is not just internal data that should be covered by data storage policies; Web content also needs to be included.
A good starting point for a data storage policy is how the data should be stored, ie on-line, near-line, or off-line, as effective archiving can dramatically reduce the size of daily back-ups. In general terms, data that needs to be edited or updated should be stored on-line, that is on devices that are part of the normal storage infrastructure, either directly attached storage, Storage Area Network (SAN), or Network Attached Storage (NAS). Data items that may need to be accessed but not updated can be stored near-line, either on disk or tape. Finally, data that is unlikely to be required but needs to be kept for legal reasons, for example financial information, can be stored off-line, on tape, and kept either on or off-site.
As a guide to developing a data storage policy Butler Group has established five steps. These are:
- Establish a data storage budget
- Assess data availability requirements
- Measure security levels
- Assess legal and governmental requirements
- Implement data policy corporate-wide
Why is it important to have a data storage policy? It is now widely accepted that the growth in data is making it increasingly difficult for administrators to manage routine maintenance tasks, such as back-ups and disaster recovery provision. This is made more acute by shrinking windows for these tasks.
There are many sources of data, and these need to be examined and included in any policies. It is easy for employees to download files from the Internet. However, the shelf life of Web documents is incredibly short - we believe that much of the information is out of date after as little as one month. Once downloaded, many of these documents will never be accessed again. Instead of downloading a document, why not just store its URL? Some companies go further and prevent employees from downloading documents at all.
Most organisations do not know what information they have. If you don't know of the existence of a document, how can you use it? With an appropriate data storage policy in place, and the appropriate software, we believe that it should be possible to catalogue all unstructured data.
Although unstructured data is the major beneficiary of data storage policies, it is also vital that structured data - mainly stored in databases - is also included. Although it is easier to manage structured data, it still needs to be incorporated into a data storage policy if databases are not to grow exponentially and become unmanageable. Simple procedures, such as archiving completed orders are relatively simple to implement, and can often be triggered automatically.
Is a data storage policy appropriate for your company? In our opinion it is sensible for every company to have some form of data storage policy, regardless of size. For small companies, this may simply involve managing the lifecycle of documents. For a large enterprise it will involve making an assessment of how data is collected and stored, and this is where we advice using Butler Group's five steps. In general terms the larger the company the greater the need for formalised policies.
The right policy?
How do you know you have the right data storage policy? The answer to this question is relatively simple. You know you have the right policy when you feel you are in control of the influx of data, your back-ups are a manageable size, and you are able to ensure that your employees have access to the correct data in the right format at the appropriate time. This will involve managing content downloaded from the Web, managing the lifecycle of documents and Web content, and archiving data that is no longer active - both structured and unstructured.
The pay off for adopting a data storage policy in our opinion is competitive advantage. The price for failing to manage data is a gradual slowdown in the flow of information throughout the company, which if left unchecked could eventually affect the viability of the organisation.
This was first published in May 2002