With 25,000 employees and 751 branches across the US, Jo-Ann Fabric and Craft Stores had its work cut out when it decided to virtualise SAP and migrate from an IBM mainframe onto x86 servers using VMware technologies.
During VMworld Europe 2012, taking place in Barcelona this week, two senior engineers from the retailer gave an overview of the project and the challenges they faced.
Jesse Drugan, lead technical engineer at Jo-Ann Fabric and Craft Stores highlights a quote from the company’s chief executive officer Travis Smith to describe the scale of the challenge: “We are going to swap out the engine while the car is still moving.”
Drugan said the company’s SAP environment, based out of headquarters in Ohio, services 23,000 users, with a BW (business warehouse) database of 10TB and an ECC (ERP central component) database of 6TB, which has 32,000 SAPS of virtual computing capacity. The environment has 400,000 SKUs with many more archived.
He said the company is heavily automated at times with 72 million RFCs (requests for comment) and 2.8 million dialogs.
Jason Nester, senior technical engineer at Jo-Ann Fabric and Craft Stores, said there were several reasons why the retailer chose SAP. These included wanting to reduce the operating system count; a need to reduce the amount of databases in use; to reduce software and hardware costs; to avoid vendor lock-in;, and he said SAP was also recommended by an IT advisory board.
“We wanted to make a move away from the mainframe,” he said.
The hardware
Drugan gave a rundown of the IBM products the company was migrating from: IBM Mainframe z10, IBM zOS 10, IBM DB2 v8.1, IBM P6 595, IBM AIX 5.3, IBM Tier 1 disk (which the mainframe was directly attached to) and IBM TSM.
“Due to the fact that the mainframe was directly attached to the Tier 1 disk, it meant it would have been very expensive to add to this. The vendor lock-in here did not allow us any flexibility,” he said.
In terms of hardware migration, the Jo-Ann Fabric and Craft Stores IT team decided to move to six HP DL580s (as VMware Hosts) and four HP DL980s (as database servers). Drugan said the company only needs four HP hosts on its DL580s but runs six to cover itself in case it loses two in an outage. On these, VMware ESXi 4.0 is run and 100 applications.
In the DL580s there are 32 cores, 256GB RAM, 10GB NICs (network interface controllers) across VMtraffic/vMotion. Drugan said the DL980s are very similar.
For storage, the firm now runs virtualised storage and three tiers of disks as follows:
Tier 1 disk (146GB, 15k RPM drives), Tier 2 disk (300GB, 15k RPM drives), Tier 3 (2TB, 7,200 RPM drives).
The software
Jo-Ann Fabric and Craft Stores went for Windows 2008 R2 Entreprise, SQL Server 2008 R2, VMware ESXi 4.0, Fault Tolerance VMs for CI servers, Dell vRanger and a multi-tier SAP landscape.
The limitations and challenges
Drugan said that VMware Fault Tolerance (FT) had a few limitations, the main being that the software does not perform snapshots: “A snapshot is like your parachute,” he said.
There was also a sizing issue as the team could only use four FT machines per host. One more drawback with the software used was that it did not include Storage vMotion.
According to Nester, the main challenge throughout the project was time, as the company had set itself a deadline of six months: “We set this time period for a reason, because in the States the holiday season starts from around Halloween and as a retailer we needed the migration done and out of the way.”
Nester said they also had to contend with code changes to support a Windows environment (Powershell) which was stressful: “I spoke to the SAP engineers and left with a notebook of notes and a box of tissues.”
The team needed to reduce the outage window for the migration to 48 hours over one weekend. To reduce this window the team spoke to several SAP consultants about tools for this process, but ended up picking a route which worked best for the company instead of following what the consultants had suggested.
What does it look like now?
Nester said that despite several benefits of migrating, the company’s patching cycles are now much longer and more security is needed than with the previous mainframe: “Everything went well, but you have to know what you are dealing with.”
The migration had zero impact on the business with only a 55-hour outage, said Nester: “We were asked why did you cancel the migration? Because there was no disruption.”
Nester said the migration has achieved annual cost savings of 80%, has removed its mainframe and P6SAS, has removed its Tier 1 disk cabinet and the firm is no longer locked into one vendor.
“We now use HP, Dell, IBM servers and the hardware is capable of 45,000 SAPS of virtual computing capacity ,” he said.
What’s next for Jo-Ann Fabric and Craft Stores infrastructure?
Nester said that next on the company’s agenda is to virtualise its database servers and implement a Monster VM with 528GB memory and 32 cores.
The company also plans to merge its SAP and non-SAP environment, to make it more simple and to further increase consumption and resource. The team will also look into a disaster recovery project, as currently this is done across country which is time consuming and costly. Nester said they will also start looking into an SAP upgrade.
Nester concluded: “VMware is a viable alternative to pricey and inflexible 'Big Iron'. Use fault tolerance to provide high availability for SAP CI servers.”
09 Oct 2012