How web giants are making datacentres lean and green

Facebook and eBay infrastructure chiefs outline their efficiency strategies at DatacenterDynamics Converged conference in London

As infrastructure managers and suppliers gathered in London to explore developments around sourcing, building, optimising, powering and cooling their datacentres, the topic of energy efficiency was notably high on the agenda. 

Ever since analyst McKinsey predicted back in 2008 that IT would surpass the airline industry as the world’s biggest contributor to greenhouse gases by 2020, those responsible for datacentres have been trying to devise ways of processing ever larger volumes of information while simultaneously reducing the amount of energy they use. 

But this is as much about greenbacks as going green – clearly, the more energy-efficient a datacentre becomes, the less it costs to run. 

Facebook and eBay, two of the biggest data handlers on the planet, understand maximising datacentre efficiency is key to their continued success.

At DatacentreDynamics (DCD) Converged, the infrastructure bosses of both businesses spoke in separate sessions about their approaches to improving efficiency. While few companies need to handle as much data or as many transactions as these online giants, the considerations they outlined are just as relevant to many less data-intensive organisations across a wide spectrum of sectors. 

Although Facebook and eBay share similar efficiency goals, the two speakers each took a very different approach to the topic, perhaps reflecting their personal management styles. 

Thomas Furlong, Facebook’s vice-president of site operations, spoke in engineering terms. He focused predominantly on how the company was leveraging its internal software expertise to improve datacentre management for maximum efficiency at a technical level. 

More on datacentre efficiency

Meanwhile, the ebullient Dean Nelson, eBay’s vice-president of global foundation services, said his primary aim had been to give senior managers simple dashboard metrics that showed how datacentre efficiency was affecting their business objectives and revenues. He said this was key to concentrating the minds of employees at all levels on what needs to be done, independent of any particular solutions and technical measures they might use to do it.

Facebook: Engineering efficiency

Furlong outlined how Facebook went through a major exercise to improve datacentre efficiency around the turn of the decade, looking comprehensively at software, servers and datacentres. Its development of the HipHop for PHP engine to speed up the execution of code, and later the HipHop Virtual Machine (HHVM), were key developments. 

“We’d have five times as many servers today if we hadn’t done HipHop and HHVM,” he said. 

The company also optimised servers for efficiency, taking out unneeded components and making them easier to maintain. At the datacentre level, it switched to evaporative cooling and improved electrical systems. 

“Together, these measures gave us about 38% more efficiency and 24% less cost in our Prineville datacentre. We’ve subsequently replicated the approach in all our buildings,” said Furlong.

These improvements prompted Facebook to create the Open Compute Project (OCP) in 2011 to encourage other organisations to join its mission to engineer “the most efficient server, storage and datacentre hardware designs for scalable computing”. With around 60 members, the group looks at improving efficiency at all levels of the stack, and between those levels. Furlong admitted that the initiative had failed to attract many European members, but hoped that would change in future.

These measures gave us about 38% more efficiency and 24% less cost

Thomas Furlong, Facebook

Today, Facebook is aiming to improve efficiency even further by gaining a better understanding of its changing utilisation levels and workloads over time, as well as more granular control over its servers and clusters so it can keep utilisation levels high while still provisioning appropriately for workload peaks and user growth. To that end the company is developing a comprehensive datacentre infrastructure management (DCIM) solution. 

Furlong said the biggest challenge for its software engineers has been to uncover and clean up the disparate sources of data needed to make the solution work effectively. “Now we are constantly adjusting and tweaking clusters to gain full utilisation,” he said. While in 2012 that involved a lot of manual tweaking, “today we can do it very quickly using software”.

eBay: Doing as the business bids

While Furlong noted Facebook had been “very focused on the tactical” aspects of improving efficiency, he admitted it was also important to figure out the business implications. This is an area where eBay has excelled. 

“How do you explain to your C-suite that you need another $100m for the datacentre? They just see it as an infrastructure overhead. We decided to create a metric that we call digital service efficiency (DSE),” said Nelson in his conference presentation.

“Our CFO cares about the cost of putting the infrastructure together, our chief sustainability officer cares about environmental impact, and the CEO, COO and the rest care about the revenue generated from this engine,” he added.

"Everyone is looking at which knobs they can turn to make the numbers better."

Dean Nelson, eBay

In 2012, eBay processed 4.3 trillion transactions for its 144 million active users, where a transaction is defined as a single URL request (including the various back-end calls made by the system in response to the original user-generated request). “We realised everything done by a user is actually consuming kilowatt hours,” said Nelson. 

By dividing the total number of kilowatt hours consumed by the total number of transactions processed, eBay was able to come up with what is essentially a “miles per gallon” measurement of datacentre performance. This top-level metric allowed the company to develop specific second-level metrics that showed how it was performing each quarter in terms of total energy efficiency, carbon consumption and revenue – all measurements that show how datacentre efficiency is directly affecting business performance in terms relevant to the various concerns of its executives.

“This promotes continuous improvement, because everyone is looking at which knobs they can turn to make their numbers better. That means you can design datacentres that are really efficient, and operate them so they continue to be efficient,” said Nelson. 

And because these metrics are business-driven, they are technology agnostic. “It doesn’t matter what solutions are underneath. We let engineers do the engineering – we just give them the parameters,” he said.

Having used DSE to guide its decisions for 18 months, Nelson said the approach had been a resounding success. Among the efficiency solutions it has deployed are liquid cooling, modular “rack and roll” datacentres, high-density clusters, solar arrays, Bloom fuel cells and onsite power generation using natural gas, as well as a host of lower-level tweaks.  

“We’ve seen a 40% increase in transactions and added 34% more servers to our infrastructure, yet we have only consumed 24% more power. This means over that 18-month period we’ve avoided using 1.6 megawatts of power that we would have needed had we not turned the knobs we did,” said Nelson.

Read more on Datacentre energy efficiency and green IT

Data Center
Data Management