Making the most of your storage resources: Improving storage utilisation

Learn about data storage utilisation and ways to improve storage utilisation rates using good design, good policies and good capacity management processes.

In today's economy, making the most of your data storage resources is a crucial part of any storage professional's job. bureau chief Antony Adshead talks with Ian Lock, storage practice lead at GlassHouse Technologies (UK), about what governs the rate of data storage utilisation, what causes poor storage utilisation rates, and ways to improve storage utilisation rates using good design, good policies and good capacity management processes. Lock's answers can also be heard below as an MP3 download.

Play now:

Download for later:

Improving storage utilisation
• Internet Explorer: Right Click > Save Target As
• Firefox: Right Click > Save Link As

Table of contents:

>>What governs the rate of storage utilisation?
>>What causes poor rates of storage utilisation?
>>How can you improve storage utilisation? What governs the rate of storage utilisation?

Lock: The rate of storage consumption is governed by several factors. Firstly, it depends on the rate at which IT projects request storage capacity. IT designers and architects request capacity from the storage team when they're putting together new solutions or extending existing ones. A busy company that is bringing new products to market will have multiple IT projects running concurrently, all generating lots of demand for storage capacity.

Secondly, storage utilisation can be affected by how storage resources are accounted for. In smaller companies without a central IT budget, capacity is usually procured on a project-by-project basis with each new project buying its own chunk of storage which typically cannot then be shared with other projects. This has a detrimental effect on storage utilization, with pockets of wasted capacity, which are very hard to release.

Large corporates are more likely to have a shared IT function with storage resources which are shared across departments with storage costs recovered by cost charging. This shared model should, in theory, drive up utilisation rates by reducing islands of unused capacity.

The third factor affecting utilisation is how quickly the applications actually fill up the capacity which has been assigned to them. A heavily used application will fill up the storage capacity allocated to it much more quickly than an application that is used infrequently. What causes poor rates of storage utilisation? 

Lock: Low storage utilisation can be caused by several factors. The first is often poor storage configuration or poor layout in the arrays. For example, where you're using RAID 1 protection or mirroring, this will cut overall utilisation rates by 50%.

Clone copies and replication volumes can also consume masses of storage capacity. Imagine a typical storage setup where a company has an array at its production site and an identical array at its second site for disaster recovery (DR) and backup. These arrays are configured as RAID 1 and every LUN [logical unit number] is replicated in the DR array.

Now imagine a 100 GB LUN configured at the production site. This LUN is configured with RAID 1 mirroring, so right away you have two copies of that LUN and a maximum of 50% storage utilisation. This original 100 GB LUN is replicated to the original DR array where it has a target LUN of the same size – so that's three lots of 100 GB. That LUN is set up for RAID 1, too, so that's four lots of 100 GB.

With a clone copy in the array at the DR site for backup that makes five copies. If that clone copy is set up for RAID 1 mirroring, that's six copies. So, in all, that 100 GB LUN which is presented to your servers is actually using 600 GB of capacity. At best your utilisation is going to be 16% and that's if your server application uses up all 100 GB, which is unlikely. This may seem like a pretty extreme example, but it's a pretty common configuration.

Low storage utilisation is also caused by overprovisioning of storage. This can happen when a project doesn't have a good forecast of the storage its applications will use. Imagine the IT department of a large bank is asked to design a system for a new mortgage product it's about to launch. One part of the solution is to be a database for customer enquiries. The mortgage department doesn't have a good idea of how the new product will take off, so it gives a rough estimate of 500 GB for the customer database.

It's very likely the application architect, who doesn't want the database to run out of space in six months time, will then overestimate the storage estimates to the storage team, so he doubles the figure and it becomes 1 TB [of storage]. That 1 TB then gets carved up and presented to the database server. If the mortgage product doesn't turn out to be as popular as expected, then that database doesn't get filled up with customers' inquiries and doesn't approach its 500 GB estimated size, never mind the 1 TB.

So, even though 1 TB was allocated to the volume, it has only filled a small percentage of it and the rest is what we call 'white space' which can't be used by another server even though it's effectively empty.

Finally, there are also examples where storage administrators keep utilisation levels low on purpose to improve performance levels. I've seen many instances of midrange modular storage arrays which have only one or two hosts attached with only 50% of their disk capacity used. This is done just to make sure there are no hotspots in contention for disk resources from hosts sharing spindles. How can you improve storage utilisation? 

Lock: There are several ways to improve storage utilisation rates. The first is through good design, good policies and good capacity management processes.

With good storage design, data is placed on the storage tier that matches its availability and performance requirements. This means data isn't stored on enterprise-class, RAID 1 mirrored storage if it doesn't need to be. Data that requires high availability will be placed on the best performing tier, while data that is accessed by fewer users and can be down for a day or more in the event of a disaster can probably be safely moved to a storage array without a cloned copy or a DR replica. Without those protection overheads, the utilisation rate of this array will probably be closer to 70% or 80%, depending on how exactly the RAID 5 groups are set up.

Good capacity management processes will help projects to not overestimate demand or stop them from reserving large amounts of capacity for projects which never happen or which have their scope changed. Good capacity management also makes sure storage is purchased at the right time, preferably on a just-in-time basis, so that large chunks of storage don't get purchased and stand idle in the data centre for months or years on end.

One technology solution that can help storage utilisation levels is thin provisioning. Thin provisioning is where a storage array presents more capacity than it actually has installed to hosts. Raw capacity is only added as it is consumed by attached host servers. It allows users to purchase less storage capacity up front, deferring capacity purchases until they are required by applications. This saves on the operating costs of keeping unused disk capacity spinning. However, the storage team must keep a close watch on the management interfaces of the storage system to ensure the disk pools are kept topped up.

The second technology I'll touch on here is data archiving. Archiving can drive up utilisation at the file system level, reducing the white space I mentioned earlier. Archiving systems keep track of file usage and can automatically migrate aged or duplicate files from high-cost to low-cost tiers. Once these files have been migrated, this capacity can be reclaimed. With a good archiving solution the enterprise array can be kept smaller in size, with aging data constantly moved off to lower cost storage systems.

Read more on Storage management and strategy