The project – which was mostly completed in 2005 – was the result of the AA's purchase in 2004 by a private equity group for £1.6 billion and the need to disaggregate its systems from that of its previous owner, Centrica, which had held the organisation since 1999.
He says, "During Centrica's ownership we had what I'd call a 'big iron' IT strategy and a one size fits all approach. Our IT costs were £64 million per year and we couldn't implement small systems. Everything cost hundreds of thousands to deploy and we couldn't react quickly to the needs of the business. Now we've moved to a situation where we determine the required service levels and decide what technical platform is most appropriate and have more than halved IT costs."
When the AA emerged from Centrica's ownership it had a mix of EMC, HP and Compaq Fibre Channel storage devices which supported 850 servers, of which about 740 were HP Proliant, 20 Fujitsu, 70 Sun, 10 HP UX and some Sequent.
As part of a year-long IT rationalisation project the firm reduced its server count to about 320 HP Proliant and Sun servers supported by a storage estate solely based on NetApp iSCSI technology. This comprises five FAS 960s with Fibre Channel disks and two Nearstore R200s with SATA disks at its primary Basingstoke site plus two FAS 3050s and two Nearstore R200s at its secondary Farnborough site. Data is snapshotted at 10 minute, one hour or one day intervals to the secondary site depending on importance.
The Fibre Channel arrays provide around 70% of the AA's storage for its core SAP, SAS, Oracle and SQL applications while the SATA-equipped Nearstores deal with solely flat file data. Total storage capacity is around 300 terabytes. Most data is saved to disk with tape used only for less important data such as Blackberry and some web server files.
NetApp's FAS devices are often classified as NAS but as well as supporting CIFS and NFS file systems they can work with iSCSI and Fibre Channel fabrics and have storage controllers that natively communicate to either Fibre Channel, SATA or SAS disks.
Saved £3m in contest between NetApp and EMCThe AA decided on NetApp technology in a run off against EMC Fibre Channel products. NetApp won out against EMC on capital and total life costs and spent £3 million instead of £6 million in the process, says O'Keeffe.
"We'd had some NetApp equipment before and decided it was the best, easiest and cheapest in terms of capital cost and ongoing running expenses for what we needed. The EMC equipment would have been twice the price and the Celerra and Clariion products were much more difficult to configure and understand."
O'Keeffe says the project has contributed to IT costs being reduced from £64 million a year in 2004 to £30 million now and has allowed the firm to reduce storage personnel from 16 to one.
O'Keeffe says, "We moved from having a number of different vendors' Fibre Channel SANs to IP SAN and NAS. Up to 16 people had been needed to manage the different types of Fibre Channel SANs, which is far more complex than provisioning a standard IP network, for which we already had Cisco networking skills. It's a lot easier to take an RJ45 lead and connect a switch to a server than to patch in through the SAN fabric with Fibre Channel cable."
Does O'Keeffe think NetApp can improve any of the products his firm has spent so much on? His chief criticism is of backup and restore tool Snapmanager when used with Microsoft's Sharepoint collaboration and content management software.
He says, "Snapmanager for Exchange and for SQL is very good, but Snapmanager for Sharepoint didn't really give us anything over other third party tools. Maybe the next version will."
He also said that NetApp's, "encryption capabilities gained by acquisition could be better integrated into products."
The AA's storage infrastructure will be due for refresh in 2009, says O'Keeffe. By that time his team will have begun work on planned data archiving and information lifecycle management projects.