"When I came to the post four years ago our central storage solution was nearing end of life," says Reeman. "We had two DEC Alphas with storage arrays amounting to 1 TB delivering productivity data to staff and students. Daily back ups were taking 25 hours, the network was maxing out delivering data and it was all very expensive."
UEA's servers all had their own directly-attached storage and IT staff were walking around all day changing back up tapes for each one. "It was an incredible waste of time and resources," says Reeman. Staff had a quota of 100 MBs of storage but this was inadequate and people were effectively opting out of the storage provided and were putting data on their C drive, which was not secure.
Given the dire situation it did not take too much for Reeman and his team to put a case together to fund an upgrade. Arguing that the existing system had gone beyond its useful life and was restricting users, the university authorities granted funding and the ICT department went out to find a SAN solution via an EU procurement process.
For core applications the DEC Alphas were replaced with Sun Solaris servers and HP Blade servers, although now they are being consolidated in a rolling process to HP blades with some Linux capability.
The university's e-science parallel processing farms crunch research numbers for its users including those in the environmental science and climate studies departments. That processing was carried out by a 100-node cluster of Clustervision 1U servers which is in the process of being replaced by Dell quad core 1U servers, providing more than 800 cores of processing power.
During the SAN procurement process UEA settled on a predominantly IBM-based offering which was implemented with integrator TecTrade. The SAN comprises IBM TotalStorage DS4500 and DS4100s, with IBM System Storage SAN Volume Controller virtualisation technology, switched by McData 4500s with Onstor SAN filers.
Daily back ups are handled by a tape library with 12 drives and 1,100 tape slots – seven of which are LT03 drives with a compressed capacity of about 800 GBs per tape and five LT02 drives with a compressed capacity of about 400 GBs per tape.
IBM was selected through an EU tender process that scored vendors on their ability to meet UEA's needs, says Reeman, with points awarded on the basis of how they measured up to specified outputs.
"We had around 20 replies, of which there were five serious contenders," says Reeman. We were surprised that IBM won out, but what they offered was the ability to create a single point, providing everything in the SAN and the ability to grow with it. It was important that we had no 'drop off point' where we would need to move to another product range or vendor."
Benefits for UEA's ICT department
While Reeman was not permitted to disclose the dollar benefits he describes them as "obvious". Compared to the previous configuration of direct attached storage it has given the university more performance and scalability as well as providing savings in space and energy use. Real savings have also been made in staff-time needed to complete daily backup, maintenance and monitoring tasks. "We had a serious problem with our existing set-up and needed to ensure we had a solution that gave us resilience and scalability," says Reeman.
But, the SAN implementation has also had unforeseen effects, says Reeman. His team originally planned for 10 TBs with the intention of expanding to 100 TBs within five years.
"We initially thought we were being optimistic with these projections but the provision of easily useable storage to users opened the floodgates," he says. "The previous set up had so restricted people that they used to write to CD or local hard drive to store data so we had no idea of their real needs. Now they have a quota of 2 GBs each and the SAN is nearing 100 TBs in capacity".
Lessons learned - understand your data and the procurement process
The entire procurement and migration process took Reeman's teams six months – with the tender process and the upgrade taking three months each. "We fell into the classic trap of taking far too long to procure the solution," he says. "There are strict guidelines to follow in EU tender processes so that non-successful suppliers can't challenge the result. It was the first time we'd done one so we were relatively inexperienced. We've been through others since and have managed to reduce the time taken to about eight weeks."
Reeman also says that if he went through the migration process again he would spend more time analysing the data beforehand. "We did it on time, with no real problems but we didn't discriminate between sizes and types of file and just did it department by department in A to Z fashion. If we had analysed files and picked out the bigger ones and dealt with them sooner we could have got some departments up and running more quickly."
The next step – a tiered architecture
The SAN implementation has proved to be just the first stage in UEA's overhaul of its storage infrastructure. The present configuration has the SAN as primary storage with tape back-up. A tiered infrastructure with archiving is the next step, says Reeman.
"It was a major step to implement the SAN and it allowed us to cluster servers and allocate quotas of storage capacity but now we need an archiving solution and to look at adding layers to our storage infrastructure," he says.
"The research groups have large data sets which they use for a while and then store for periods before dusting off again. We haven't aligned our storage infrastructure to that way of working so we need a secure archiving facility, probably using IBM Tivoli Storage Manager HSM," adds Reeman.