There is good news and bad news for anyone considering or already pursuing a career in data storage. There is also some mixed news, depending on whether you relish new opportunities or prefer to stick with what you know.
The good news is that since the emergence of Fibre Channel in the late 1990s, the options - and related complications - around configuring, connecting and managing data storage have increased. So much so that research conducted by the Storage Networking Industry Association (SNIA) found that 55% of European user companies believe they need a dedicated storage team. The bad news is that 45% do not see such a need.
The mixed news is that IP (Internet Protocol) networking, infrastructure management software and other developments are together opening the prospect of storage becoming just another part of the company network based on industry standards, thus reducing the need for storage staff.
The opportunity here for optimists in data storage is that they could expand their careers into networking.
Data storage is relatively new as a career option, says Bernard Zeutzius, EMEA product manager at Cisco. "Fibre Channel created the new job of storage networking manager in the late 1990s. Suddenly, instead of having to have all the servers and storage in the same room, you could connect them over a distance, and remote servers could share central or distributed storage resources.
"But you now needed people who understood the options and the architectures and how best to implement them depending on the applications, the preferred suppliers and other factors."
The take-up of Fibre Channel and storage networking meant a shortage of specialists that has never totally been met, says Stephen Watson, Storageworks manager at Hewlett Packard in the UK. HP sees the problem as so serious that it is putting £3m into a new scheme to train staff at its resellers.
"Six years ago I was working for a reseller and there were only 82 HP storage specialists in the UK. You would see the same people at every storage event," Watson says.
"We need specialists who can get user companies thinking about things like redundancy, replication, disc mirroring and whether to have a central consolidated file system."
Staffing issues are such that they are influencing users' decisions when they look at data storage, according to Ken Kaban, European technology manager at Australian multi-beverage company Foster's Group. "We were struggling for disc space - like everyone else - and needed a new system, but we could not spend time learning about and implementing Fibre Channel switches and then diagnosing fibre issues.
"The impression you get of fibre is that you need a network engineer with a Harvard degree and a PhD to maintain it. In ideal circumstances it works fine, but then comes that Sunday when something causes a little bit of fibre to heat up and your data gets corrupted. We have seen our colleagues at the Foster's headquarters in Australia facing these problems."
These skills considerations led Kaban to choose a storage area network (San) hardware and software package from EqualLogic. This needed no new skills, says Kaban. "It is out of the box and up and running in 10 minutes."
Such a package might not be suitable for much bigger organisations, but the availability of products like this underlines the variety of options that has grown with the emergence of Fibre Channel and other developments, such as IP networking, internet Small Computer System Interface (iSCSI) and Serial Advanced Technology Attachment (Sata). All of these have come in a relatively short time and all are evolving fast.
"Two years ago suppliers said cheap Sata discs were just for PCs and were no good for enterprise use, and now we have Sata discs holding a terabyte," says Zeutzius.
"People say iSCSI is okay for small and medium companies, but Fibre Channel is mandatory for large enterprises. However, a 10Gbit iSCSI storage interface could change the way people look at storage networking."
All this means data storage can provide a satisfying career, although it may not appear exciting at first glance, says Watson. "How can you get excited about something that apparently just sits in the corner and does not do anything? IT people have perhaps focused on some of the apparently more dynamic things like the rapid progress in processor speed, server virtualisation and the potential of IP.
"Ideally, you actually want storage to stay boring because if it goes wrong the business impact can be dramatic. But to prevent it going wrong you need people who can understand it, devise the right strategies and implement and maintain the right systems," Watson says.
"Storage is a satisfying topic in other ways too. Specialists can streamline the equipment in datacentres, which means they reduce the power requirements and have an impact on the environment and on company costs. There is definitely a career in this field."
However, there is no clear career path to become a data storage specialist. Indeed, the job does not feature in the National Skills Framework of the Information Age, which defines IT roles.
"Storage is an area that people tend to specialise in after working their way up through the IT ranks," says Dave Jolley, a data storage recruitment consultant at IT recruitment company Greythorn. "This means training and experience are often gained on the job, with professional certifications coming later.
"Specialists need good network-based experience and often a technical background in testing or support. In terms of soft skills, attention to detail and good communication on a specialist technical topic are a must."
This view of a non-specific path into a data storage career is confirmed by Declan Van Esbeck, managing director of storage systems specialist Redstone Technology.
"Our engineers will have gained an appreciation of the concepts over two or three years of implementing and managing standard server Nas systems. Strong site experience is invaluable. An understanding of storage provisioning and experience of a major infrastructure management software tool, such as EMC's Legato or HP's Openview, is also essential."
Van Esbeck highlights issues surrounding proprietary products and the problems of cross-training between some of them - issues that are also underlined by the SNIA's latest annual survey of end-users.
"More than 60% of users would be willing to pay more for a tested and supported interoperable multi-supplier system than for a single-supplier proprietary system," the association says.
"Barriers to new technology are seen as greater than in the previous survey. These barriers are interoperability, heavy reliance on supplier roadmaps, and significant certification and testing."
Clearly people looking at data storage as their next step need to be self-starters, prepared to get the right experience and knowledge themselves. And the rewards are worth it.
"Storage is vital in all enterprise systems, so demand for experienced data storage engineers is rising, and consequently salaries are high, especially within the M25," says Van Esbeck. "A senior consultant could expect about £60,000 a year."
Jolley agrees. "Experienced permanent staff in a user company could be on £45,000-£50,000. Contract staff could expect £200-£220 a day, even at junior level. But staff need to be prepared to work shifts or be on call 24 hours a day if IT operations run around the clock."
All this has evolved in the 10 years since Fibre Channel emerged, and the next 10 years could be very different again. "There has been a real border between the storage networking team and the traditional networking team, but there is the interesting possibility of that barrier falling," says Zeutzius.
"The storage manager has seen his field as one that others do not understand, and has guarded his responsibility for the organisation's data, arguing that if that fails, the organisation fails.
"However, standards are now well established, notably IP, and the software tools are much better and easier to use than 10 years ago, so IT departments can now think about making storage networking just another part of the job definition of the networking team.
"The border between the two has been blurred by storage specialist EMC's takeover of Legato and other software companies, giving it general systems management tools.
"You still need someone to define storage needs and talk to suppliers, but when it comes to connecting things together over distance and managing the service, why not use people who have always done this: the networking team? Already some big organisations have done away with a separate team."
Zeutzius does not see this becoming the norm overnight, but he points again to rapid developments in related areas such as Sata and the use of IP, which could quickly bring compelling arguments around costs, consolidation and simplification.
Data storage specialists can look at this not as a threat, but potentially as an opportunity to expand their horizons into networking, he says.
Whatever the future, there is general agreement that data and its storage will remain vital issues for organisations of all sizes.
"Companies are storing more data than ever before, and there will be no shortage of jobs for data storage specialists for some time to come," says Jolley.
Whether those jobs will ultimately be in data storage or networking remains to be seen.
Comment on this article: firstname.lastname@example.org