How Experian is retro-fitting its datacentre to boost efficiency
Experian's Fairham House datacentre in Nottingham uses blade servers, virtualisation and cold aisle containment to improve energy efficiency.
To most people, Experian is a credit rating agency. Consumers only hear about the company when they are turned down for a loan or credit card. But the firm does far more than this – it is a data company. It provides analytics to banks, marketing services to help retailers understand customer demographics, and has a part to play in fraud detection. Its datacentre knows a lot about you.
Experian operates in more than 50 countries globally and runs twin datacentres in Nottingham, Texas and Brazil, along with a number of datacentres in Asia. Barry Westbury, operations director, global technology services at Experian, is responsible for managing the two established facilities in Texas and the UK.
“Historically, from an IT perspective Experian was regionalised, but since 2008 it has been on a journey to globalise," he said. "One of the first tasks was to globalise the network. It sounds easy, but it is a challenge for a company that was previously regionalised.”
The Fairham House datacentre in Nottingham is configured in a similar way to Texas, with a back-up facility 12 miles away linked by diverse 640Gbps dark fibre, running left and right out of the building. “We have tried to eliminate any single point of failure in any of the Experian infrastructure,” said Westbury.
Fairham House is the main production centre in the UK, representing a £36.1m investment for Experian, a purpose-built datacentre constructed in 2004. It is a single-storey, out-of-town facility occupying 78,000 square feet with 16,000 square feet of expansion space. As well as IT redundancy, the facility runs four generators, four chillers, and multiple power feeds into the site.
Security is also built around multiple layers of redundancy. For instance, Experian runs two firewalls and several anti-virus systems from the major providers. Experian has not skimped on physical security either. Taxis only stop outside the site, which is located in an industrial park about 20 minutes’ drive from the city centre.
The building is fitted with blast-proof glass, motion detectors surround the building, and a moat was dug on the site for drainage to prevent intruders breaking in through the sewage system. Only pre-announced visitors can enter through the front entrance. The sunken reed bed is purely on the front of the building, and provides security from the road side. The other three sides have earthworks to increase physical security against vehicle penetration. They are also protected by the tremor fencing which also has an infrared beam around it.
The dimensions of the building are similar to the Texas site, which was built two years earlier, said Westbury. However the UK site, being two years younger, is taking advantage of some of the newer developments that were available at the time of construction. “If we did it again tomorrow, we would certainly do it differently,” Westbury conceded.
The control room runs 24 hours a day, 365 days a year, with three shifts of staff. At peak times, up to 20 people monitor IT systems and facilities systems from the control room using Tivoli as the overall system management tool, which manages other tools.
Green datacentre initiative
Experian’s credit bureau runs on a new IBM z10 mainframe. The site also runs 3,100 servers, which increasingly are being migrated to virtual servers. Experian uses HP C-class blades. “We have a deployment model that stipulates running virtualised applications wherever possible. If it cannot be virtualised, run it on a blade, and if it cannot be run there, then run the application on a physical machine,” said Westbury.
“Server virtualisation is key to us. We run 1,900 physical servers and 1,200 virtual servers.”
The 19in pizza-box-shaped servers that Experian used in the past consumed 900 watts of power, while a virtual machine only needs 25 watts.
The site uses cold aisle containment. “Instead of wasting 60% of your [cooled] air, cold aisle containment prevents air from escaping," said Westbury.
Another strategy used by Experian is slowing cooling fans down by 20% – running at 80% of maximum speed uses 50% less electricity.
Before cold aisle containment was installed, up to 55% of cold air generated by the air-conditioning units was wasted. "Now we only generate the minimum amount of air necessary and it is being directed into the server racks," said Paul Dennis, technical facilities manager.
Experian is now running the fans at 60% of their maximum speed, and looking to reduce this further.
“We are working on an experimental system to further reduce the speed,” he said. “The system uses a huge amount of chilled water that was originally chilled to six degrees Celsius. But we have increased that temperature and we are now running the chilled water through the cooling system at 9.5 degrees Celsius.”
Experian found that this reduces the load on the chillers, leading to a reduction in electricity consumption.
At the same time, Experian is increasing the air temperature above the servers. It wants to increase this temperature to 27 degrees Celsius, as recommended by Ashrae, the American Society of Heating, Refrigerating and Air-Conditioning Engineers. Currently, the average temperature is at 22 degrees Celsius, but on Computer Weekly’s site visit, from a purely subjective basis, the server rooms felt warmer, near 25 degrees Celsius.
Ultimately, Experian is investigating the potential of installing of "free cooling" technology at Fairham House, either using cold external air to cool the server equipment directly, or to pre-cool the chilled water system.
Outside traditional datacentre metrics, the site is also tackling its carbon footprint by minimising paper waste. The company has centralised printing – when a user prints a document, it is only printed once activated by a user using a swipecard. The printout is automatically routed to the printer the user has activated.
From an energy basis, there is no wastage of printer paper and the site needs 30 less printers.
When Experian began its green IT initiative, its PUE (power usage effectiveness) was 2.0. Thanks to virtualisation, cold aisle containment and slowing down the cooling fans, Fairham has achieved a PUE measure of 1.53 and is working to improving this to between 1.3 and 1.5.
Photography by Cliff Saran