Big Data: Utilities rise to the smart meter challenge

The mass of information from smart meters is leading utility suppliers to reconsider how they use their data

In March 2011, the UK government fired the starting pistol in what is set to become a race to transform data management in the utilities sector. The Department of Energy and Climate Change mapped out plans for the installation of 53 million smart meters in 30 million homes and businesses by 2020.

Smart meters offer households and businesses the chance to understand and reduce their energy usage in much greater detail than previously possible, when meter readings were taken once a quarter, or even annually. They promise to help utility firms improve the accuracy of billing and cut visits to properties to read meters.

Given that utility firms have commonly struggled with the accuracy of their customer and billing data, the prospect of mass smart metering could present a considerable challenge, said Stuart Ravens, principal analyst, energy and sustainability technology with Ovum.

"At the moment we could have a meter reading every year. With smart meters, if they read every half hour, that’s 17,500 meter readings a year. When we talk about ‘exponential’ growth of data, that is really exponential. And that is just considering the billing side," Ravens says.

As such, utilities at first found scaling smart meter management systems difficult, he says. As not every meter message gets through to the back-end server, data validation can place a substantial I/O load on servers, Ravens says.

Forecasting energy usage

The ultimate goal of smart metering is to allow utility firms to forecast energy usage, to improve their performance on the settlement markets – where money can be lost through inaccurate predictions – and to match supply and demand more closely.

The utilities are not new to analytics and forecasting. They have been used on the operations and maintenance side of the industry for years.

Big Data at Yorkshire Water

Yorkshire Water, a water supplier and waste management firm which supplies around 1.4 billion litres of water each day, has been using analytics to forecast flooding and pollution in its network of 62,000 miles of water and sewerage mains.

Yorkshire Water senior IT professional John Samson has helped implement the systems which enable water engineers to predict when damaging pollution is likely to take place in the sewerage overflow. Combined sewer overflows are designed to allow water to escape during times of heavy rainfall, when pollutants are diluted. Discharge during dry weather can be more serious.

Data mining tool IBM SPSS Modeler provides users with daily spreadsheets which forecast the overflow sites likely to develop blockages. Operators can then better target engineers’ visits.

"This is an extremely simple application and I think that is the strength of it – there is no complex user interface they have to worry about. They just pick up their spreadsheet and apply their own local knowledge and experience," Samson says.

The system is now being developed to incorporate Met Office rainfall data in near real time, using an OSI Pi interface. This allows the system to build models which apply in wet and dry weather.

"We are going to turn the dry weather model into an all-weather model that will double its usefulness," Samson says.

Benefits of integrating data

Other examples of the successful applications of data analytics in the sector come from the customer-facing side of the business. EDF Energy has been using a SAS data management and predictive analytics platform to cut the number of customers leaving the utility firm.

While customer-facing and operational data systems remain distinct, however, utility companies may fail to exploit the opportunities that smart metering can offer by integrating these two sides of the business, says Roberta Bigliani, head of IDC Energy Insights for Europe.

"We could be in a situation where we are creating silos of data rather than making more consistent availability of the data," she says.

Meter data could help fraud detection, predict maintenance requirements and eventually lead to smart grids which respond intelligently to variations in supply and demand.

"To do this the data needs to be validated and translated into a metadata model, to create something that is usable by multiple applications," Bigliani says.

"IT people need to work with the line of business to define a master data sort of approach and try to create a layer where all the data coming from meters or operational systems, are transformed into pieces of data that different applications can call."

This is the Holy Grail for utilities. By offering tariffs which discourage consumption while supply is weak, and releasing stored energy in anticipation of peaks in demand, utilities can avoid investing in production which matches only the maximum levels of demand. It is also part of the transition to alternative energy sources, such as wind, solar and tidal energy, which exhibit fluctuations in the energy they provide.

Projects such as the Eco Island on the Isle of Wight strive to demonstrate that smart metering and analytics can become essential tools in forecasting variations. It is a core part of the effort to make the island self-sufficient in energy.

This project is a picture of where the utility sector could be heading, albeit on a smaller scale. The lessons learned from projects like this could provide important insight into the data challenges involved in making the leap to new sources of energy on a national or international level.

Read more on Business intelligence software