At the end of 2001, Computer Weekly and specialist storage consultancy Posetiv surveyed more than 600 senior IT professionals in IT departments across the country to discover how they were dealing with the data explosion.
Then at the end of 2002 Computer Weekly and Posetiv repeated the exercise, this time involving more than 1,000 senior IT professionals. The results confirm that IT managers are still wrestling with the same storage issues. But a growing proportion are adopting new networked storage architectures to help bring the storage hydra under control.
Now, as in 2001, the overwhelming majority of respondents to the Computer Weekly/Posetiv survey say they are experiencing growing demand for data storage. A massive 96% of those surveyed reported that their storage capacities were increasing - exactly the same figure as in 2001. And once again, more than half of the respondents reported that demand was being driven equally by existing applications requiring greater storage capacity and new applications generating fresh demand.
As in 2001, the single biggest application driving this growing storage demand is messaging. Almost 70% of respondents cited e-mail as the biggest culprit. Here, IT managers are being hit by a double whammy. The volume of e-mails is still growing rapidly, but the average amount of data in each e-mail is also increasing as users adopt more data-hungry formats.
Messaging is closely followed by general office applications such as Word and Powerpoint, identified by 44% of respondents as major storage-capacity guzzlers. In third place, cited by 27% of respondents, are internet-related applications such as streaming video and voice technologies.
The fact that e-mail is by far the biggest driver of the growth in storage demand underlines the central role this application now plays in business processes. Experts say it also shows the importance of developing a strategy to tackle the avalanche of e-mails being sent and received by organisations.
Zarah Damji, research manager at independent analyst firm IDC, believes that too few companies enforce limits on their employees' e-mail accounts. "E-mail is a huge headache - a lot of companies don't have any strategy in place for coping with e-mail capacity thresholds." she says. It's not something that companies in the small-to-medium-sized area tend to do systematically, she adds.
However, recent legislation means businesses are under increasing pressure to archive their e-mails for legal purposes. So IT departments are having to do some hard thinking about how they manage their storage.
It is, perhaps, not surprising then, that when senior IT professionals were again questioned about their biggest storage-related concerns, the results were strikingly similar to those in 2001. Some 66% of those surveyed said that the biggest cause of sleepless nights was still managing storage, compared with 70% in 2001. The second biggest concern, as in the last survey, was the cost involved in managing storage.
Experts say these concerns emphasise the importance of effective storage networking technologies such as storage area networks and network attached storage (Nas). Graeme Rowe, marketing director at Posetiv, explains, "Managing storage is a major issue and that's why people are looking to centralise by putting in a San or a Nas - use of these two technologies has gone up in the last few years."
Certainly, the most recent research suggests that IT managers are becoming increasingly familiar with San and Nas. Some 44% of respondents to the 2002 survey said that they were already using or planning to use a San, compared with 35% in 2001.
Use of Nas is also growing, with penetration levels higher than Sans. Almost half (48%) of IT professionals surveyed are either using or considering the technology, up from 39% the previous year.
Rowe believes that centralising storage in this way makes good sense on the part of IT managers. He says, "If everything is in one place it is easier to manage."
San and Nas, both comparatively new technologies only a few years ago, are becoming increasingly mainstream. The figures from the Computer Weekly/Posetiv survey are in line with those from research conducted by IDC.
"This shows that, despite the [economic] slowdown, people are still thinking of networked storage solutions," says IDC's Damji.
However, for those companies already using a San, the research from Computer Weekly and Posetiv reveals the same problems resurfacing. As in 2001, lack of skilled people was the biggest problem organisations with a San faced in 2002. Some 40% of San adopters cited this as an issue in the most recent survey, compared with 46% in 2001's study. Once again, this was closely followed in the league table of San headaches by the initial cost of building a San.
With pressure on capacity showing no sign of abating, it comes as no surprise that spending on storage is expected to continue to rise rapidly over the next 12 months. Some 68% of those surveyed foresaw a storage spending increase in 2003, with only 4% expecting investment to decrease. More than half said spending on storage would grow by more than 5% in 2003, and one in 10 expected to increase spending on storage by more than 25% this year.
Spending on storage has been growing for the past decade. But users have been changing the way they spend this money. Whereas in the late 1990s the key challenge for companies building storage infrastructures was the acquisition of hardware, there is growing emphasis on buying software to squeeze better use out of storage hardware.
Rowe says, "We have seen in the last few years more and more pressure from the business telling the IT department to justify its spend on technology. San and Nas are an element of this, but you have to put in effective storage management tools to increase the utilisation of what you have got."
According to Rowe, a growing number of IT managers are looking to storage resource management (SRM) software to enable them to see who is using what storage, where and how. One of the key benefits of SRM is its ability to automatically react if it identifies a storage management problem such as a user running out of disc space.
He predicts, "The one big issue in technology that will be at the forefront this year is storage resource management solutions such as those offered by Fujitsu Softek and Tivoli."
For Damji at IDC, the question of how to deal with storage management falls into two parts. She says, "There are two ways of dealing with storage management - you can tackle the specific problem, such as e-mail, or implement a wider solution, such as SRM."
SRM, however, provides the most comprehensive answer to the question of storage management, according to Damji. She says, "SRM is a much wider solution - it can generally be used to implement better policy-based storage management for all sorts of applications, not just e-mail."
One storage technology that has been hitting the headlines over the past few years is virtualisation, which enables users to treat their entire storage infrastructure as a single storage resource, as opposed to a plethora of individual hardware devices. Rowe says, "Virtualisation allows you, if you have got IBM and EMC storage devices, for example, to link these devices together and let you see one storage pool."
Rowe believes that users could resolve their storage management headaches by making use of both virtualisation and SRM. "It's a very shrewd company that would consider both SRM and virtualisation to get the best out of their storage infrastructure," he says.
However, the latest Computer Weekly/Posetiv research reveals that less than a quarter of organisations are planning to implement virtualisation.
Only 23% of respondents said that they were planning to introduce the technology in the next two years, while a massive 43% had no plans to introduce a virtual system whatsoever. Perhaps the most surprising fact, however, was that 34% of respondents were not even aware of what a virtual storage system was.
Virtualisation is, admittedly, a relatively new technology, albeit one that has received considerable press attention over the past year. For her part, Damji believes some companies may have been flummoxed by the confusing messages they have received. "There has been a lot of hype about virtualisation," she says. "That has created a lot of confusion in the marketplace, and there are also a lot of different solutions out there."
Despite the fact that storage is becoming a bigger issue for most organisations, few have access to specialist expertise. The overwhelming majority of firms are still relying on general in-house IT expertise to make key storage decisions. In the latest Computer Weekly/Posetiv survey, 74% of companies reported that they were using their own general IT expertise to plan their long-term strategies, down only 3% from the 2001 survey. In an identical result to 2001, only one firm in 20 is making use of an outside storage specialist.
Companies do appear to be paying greater attention to measuring the return on investment they get from storage. As in 2001, only a small proportion of firms are measuring their storage ROI separately, with just 8% examining storage as a distinct entity from their overall IT investment.
However, the latest research does reveal an increase of 10% in the proportion of companies that measure storage ROI as part of their overall spending on projects and systems. This may be partly the result of the belt-tightening that has followed the global economic slowdown, although taking a more strategic view of their storage can only help firms tackle longer-term issues.
On the strength of the past two surveys, the hot topic for the next few years will continue to be storage management. If companies don't get a grip of their existing storage, they will only face a tougher time in the future.
After all, it seems unlikely that the amount of data produced by companies will decrease over the next few years. On the contrary, we are living in an increasingly data-intensive age, so the volume of information used by firms will continue growing. IDC's Damji says, "If companies don't get effective storage management in place, then they could just end up spending a lot more money on hardware then they need to." The focus is already moving onto software and its functionality, she adds.