With e-business and compliance inflating storage requirements, network attached storage and storage area networks are evolving to help businesses scale up.
The number of transactions channelled through the internet is increasing daily.
Web-based transactions and e-mail have to be stored and managed, which means that a lot of these companies and branch offices are finding their traditional storage systems have to be increased more often and managed more rigorously.
As this influx of data increases, company managers are finding new ways to analyse the information, which creates even more data. To add to the problem, the deletion of files is also becoming less of an option as regulatory bodies begin to dictate data retention policies.
The challenge is to find some way to scale up storage in an elegant fashion rather than just hanging more discs onto every server.
Pat Lee, senior product manager at SME data storage specialst EMC Insignia, said, "About five years ago people had a reseller helping them install a server that had probably 80Gbytes of SCSI storage. In the smaller business of under 50 users this may have held their mail server, file server and database server.
"Today, they would have replaced the server with a bigger server, or several servers, directly attached to yet more storage. Even then, with the exponential growth in data this would soon become limiting.
"The reseller or value-added reseller does not want to have to tell the customer to buy yet another server because the current ones cannot take the loadings, so they are looking at other ways to increase storage."
Lee suggests that it is time to switch from direct attached storage and make disc space available through a network attached storage (Nas) appliance or to consider a storage area network (San).
Robin Burke, research vice-president at Gartner, said that such a strategy could not only make storage more flexible but more economical. "With direct attached storage or with just a bunch of discs you are probably only using about 35% to 45% of the available storage space.
"When you go to a San or Nas you have the ability to increase that efficiency to 80% to 90% without too much trouble - assuming you set it up right. For small-to-medium-sized enterprises who are typically watching their budgets, it allows them to make more efficient use of the storage they own."
The advantage of network-attached disc systems is that the unpredictable growth of e-mail storage and databases can be managed much better and with greater reliability. The problem is how to sell the idea to the business.
Lee has seen growing interest in entry-level storage systems. He said businesses do appreciate that when a disc fails it should not bring down the system, and that the system will scale up as its needs increase. "In companies of over 100 users I am starting to see more penetration of entry-level Sans, and we are starting to see the penetration of entry-level Nas boxes at the lower end of the market with upwards of 20 users."
What appears to be happening is that the major suppliers are selling a full range of storage products, from entry-level Nas to ultra high-end Sans.
Graham Titterington, principal analyst at Ovum, has observed this change in the market. At one time the Nas and the San camps were very separate. "A few years ago if you wanted Nas you went to Network Appliance and if you wanted San you went to EMC. Now EMC also sells Nas and Network Appliance offers Sans.
"The market is much more blurred and everybody seems to be selling everything. We are even seeing the growth of mixed Nas and San systems," he said.
The ability to mix or link Sans and Nas is primarily due to the development of iSCSI (Internet Small Computer System Interface). This brings Ethernet-based standards to what was previously a proprietary protocol market.
The ability to use IP networks simplifies installation and reduces installation costs. "iSCSI is cheaper, more flexible and more accessible. You may not get the same bandwidth, but there is plenty of headroom for most people," Titterington said. "You are not going to buy Nas if one disc will do, but if you need to move to the next level for scalability you have to consider the network technology - and that is invariably IP so iSCSI offers the simplest route."
The arrival of iSCSI has also increased Microsoft's interest in supporting network storage. Ewan Dalton, technical architect at Microsoft, said, "Historically, the official Microsoft position was that San and Nas systems may well work with Exchange but it was not a Microsoft-supported configuration.
"That changed about two years ago when the iSCSI standard started firming up. As a result we now have specific support for quite a number of iSCSI-based storage devices."
Dalton emphasised that support is only offered if the hardware used is certified by Microsoft's Windows Hardware Quality Labs. Microsoft cannot be sure that all storage systems implement the controls that ensure that discs are written in the way that Exchange expects it to be done, Dalton said.
"We have to be confident that a system works as advertised and therefore is not going to corrupt data. Exchange is pretty I/O intensive and we do not know whether a cheap adaptor is going to mangle that data or not.
"If you go for a higher-end iSCSI card that has been through the Windows Labs testing, we are confident that the hardware will be reliable under stress and that the drivers that support it have been written in an appropriate way."
Despite Dalton's caveat, Exchange is being used on non-certified systems quite successfully. The main point is that Microsoft will not take responsibility if things go wrong on one of these systems. This would be a problem if a partner company insisted on best practices to satisfy its compliance requirements.
A lot of the compliance demands that have hit larger enterprises do not impinge directly on SMEs. However, SMEs often want to do business with large corporates or public sector organisations.
And according to Hugh Jenkins, enterprise marketing director at Dell UK, this will result in a trickle down of compliance requirements, as inter-supplier agreements and contracts increasingly demand that businesses in their supplier networks cover themselves by improving their ability to archive and retrieve e-mail transactions.
Jenkins predicts that the move of e-mail from the personal communications sphere into the business world will increase the desire for reliable, scalable storage.
"As a society we are starting to get more litigious and if you do get embroiled in a legal case the things that are going to get looked at are the e-mail conversations that took place between the disputing entities. So you will have to have that stored and easily retrievable.
"Larger organisations know this and many are being made to comply. Small but forward-thinking organisations will ensure they are able to play on the same playing field, especially if they are part of the supplier network."
According to figures from analyst firm IDC, Dell is the fastest growing supplier in the Nas market, with an increase in revenues in the fourth quarter of last year of 65%.
Jenkins said some of the drivers of this growth have centred around e-mail, which is obviously creating a need for more storage but also fuels demands for greater control of that data.
"There is a growing realisation that the e-mails have to be stored securely and not left on desktop systems. There needs to be a centralised mechanism where it is regularly archived and stored away so that it can be retrieved in the face of either litigation or increasing compliance requirements," he said.
Both San and Nas appliances can help because they treat their attached discs as though they are a single storage space. Data is stored wherever space allows and new discs can be hot-plugged into the array without having to bring the system down.
Where the two systems differ is in the way data is stored. Nas is just like a conventional disc system storing everything in a file format. San technology was designed more for database-like content and works on a more transactional basis by storing blocks of data, such as a single record in a database table.
A major advantage of centralising data storage is the ease with which data can be shared. In the same way that collaborative software is helping to create new working environments, the data-sharing possibilities of Nas and San systems are allowing more departments to share information for analytical purposes.
Now suppliers are looking at combining Nas and San systems to gain the best of both worlds. This is currently targeting high-end users but change is coming to the lower end.
Paul Hickingbotham, solutions manager at storage supplier Hammer, said, "iSCSI is a very good way to allow San and Nas to be mixed in a single solution. But it is at a critical stage where, although we are seeing a big increase in iSCSI sales, it is not yet fully adopted and trusted as an architecture."
He expects 10Gbyte network hardware to start to become more affordable and attractive to smaller businesses later this year or early next year, but said there is a lot of sense in using iSCSI on current networks.
Although a Fibre Channel San is a very good idea for high-end data centres it is still too expensive for the SME, especially when adding a back-up strategy. In Hickingbotham's experience, the traditional San and Nas technologies do not really help address this, whereas IP systems such as iSCSI mean the technology can be treated in a similar way to SQL databases and their disaster recovery strategy.
The use of Nas and San in smaller business is still in its infancy and Gartner said it falls below their radar. But this will soon be changing.
Burke said, "If you look at the key players in the storage market you find that about two years ago the light clicked on and almost everybody announced programmes at the same time to target the SME market. They had run out of expansion in the high and mid-ranges and had to increase their market scope.
"Now they are pretty active and I guess once we have seen a few more quarters of the low-end products we will be able to get a better feel for it."
Glossary of storage terms
A server that is dedicated to file sharing, Nas does not provide any of the activities that a server in a server-centric system typically provides, such as e-mail, authentication or file management. Nas allows more hard disc storage space to be added to a network that already utilises servers, without shutting them down for maintenance and upgrades.
With a Nas device, storage is not an integral part of the server. Instead, in this storage-centric design, the server still handles all of the processing of data, but a Nas device delivers the data to the user. A Nas device does not need to be located within the server, but can exist anywhere in a Lan and can be made up of multiple networked Nas devices.
Storage area network
A San is a high-speed sub-network of shared storage devices. A San's architecture works in a way that makes all storage devices available to all servers on a Lan or Wan. As more storage devices are added to a San, they too will be accessible from any server in the larger network.
Because stored data does not reside directly on any of the servers on the network, server power is utilised for business applications, and network capacity is released to the end-user.
iSCSI (Internet Small Computer System Interface) is an IP-based standard for linking data storage devices over a network and transferring data by carrying SCSI commands over IP networks. iSCSI is important to San technology because it enables a San to be deployed in a Lan, Wan or metropolitan area network.