Using public sector open data to benefit local communities

Tech City’s Flood Hack shows how open data can help communities - an aspect attracting interest in Leeds council and other UK cities.

This article can also be found in the Premium Editorial Download: Computer Weekly: Boost productivity with collaboration technology

Tech City’s Flood Hack shows how open data can help communities - an aspect attracting interest in Leeds council and other UK cities.

In February the Environment Agency became a real-time advocate for open data – whether it wanted to or not. During the flooding crisis – the UK battered by storms and many forced out of flooded homes in the south-west – the Environment Agency and government called on developers in Tech City to help.

The government organised a "Flood Hack" event, where developers used newly available government data – including flood sensor readings updated every 15 minutes – to create digital services, which in turn increased demand for data. In the past, the Environment Agency’s attitude had been to charge a licence fee for access to its data but, during the Flood Hack, the agency freely released datasets for developers to create applications

After the event, the Cabinet Office embarked on a project to find the best way to ensure access to information of this type in future.

The Open Data Institute’s technical director, Jeni Tennison, says the Flood Hack event demonstrates the importance of releasing data. She says that, if the data had been released earlier, it may have been possible to create applications to plan for the disaster ahead of time.

But, she says, there is "closed-mindedness" in how this flood data could have been used and there has long been a debate about releasing it: "The Environment Agency thinks the only way the data can be used is to put it on a map," she says, "and they’re doing that anyway, so why would anyone else need it? “Open data is transformative. It can be used as a tool for any kind of policy purpose. If you have a policy about protecting people from flooding and knowing where to build flood defences and making people prepared, then one of the things for you to do is to release data and information to make sure it’s available.”

Open data and Whitehall

Government is beginning to understand the importance of making data freely available. Cabinet Office minister Francis Maude talks of open data as the next industrial revolution. "Transparency, accountability and citizen choice" has been the government’s mantra for open data since first making public sector datasets available. But there is also hope that, by doing so, it will help boost economic growth, with small businesses in particular taking the data and creating applications. In a recent Computer Weekly interview Maude said: "In the Cabinet Office, Number 10 and the Treasury, we have been vigorous in promoting open data, but have found it heartening how many other departments are on board. When I urge colleagues and officials to go further, yes we are trail-blazers, but we are often pushing at an open door. The Home Office crime map app has created something which is world-leading."

But while it has been bigged-up in parliament, the message has failed to trickle down to the civil servants on the front line. Recently, the government announced a multimillion-pound fund to invest in organisations championing open data. The first £1.5m grant will support projects working on releasing public data as well as those training civil servants in their understanding of open data. The Open Data Institute (ODI) will receive £150,000, part of which will be used to train dozens of Whitehall staff to use and understand open data.

Government officials need to understand they can use open data as a tool

Jeni Tennison, technical director, Open Data Institute

"Government officials need to understand they can use open data as a tool," says Tennison. She says most think open data is something to be done for the sake of transparency. But there are still a number of civil servants championing open data. There was a lot of demand from civil servants for training in this area but it was impossible until the recent funding announcement, according to Tennison. Part of the ODI’s Whitehall training is to explain the clear benefits, and to help them think


Attitudes towards open data need to be championed in a more localised fashion, says Emer Coleman, business development director at TransportAPI, a startup developing an open platform for transport data. The greater economic value will come from local data: "The real benefits are at city-level rather than national," she says. If local authorities share their data, developers could produce innovative applications that, in turn, benefit the council and its citizens. "You never have the same tech experience internally as you have in the market."

Coleman says freeing the data allows innovation to happen, adding that, were a local authority to keep hold of its data and make its own application, it would "kill the innovation". Transport for London (TfL) is an example of innovation with open data for bus and Tube travel in the capital. "TfL doesn’t have its own application, and there are lots of apps made by developers – if TfL had gone out with its own app, there would have been no-one to compete with it," says Coleman.

Making the data freely available prompted developers and companies to come and improve consumer choice and drive competition in the market, she says.

Local authority concerns

Some cities – such as Manchester, Leeds and Birmingham – are forging ahead, but others have not advanced so far down the open data path. Tennison says local authorities are concerned about the costs of publishing and maintaining datasets, and councils face difficulties in managing procurement and supplier relationships.

Tech city app developers meet flood challenge

From the government’s "Flood Hack" event flourished a multitude of apps that monitor data about floods from diverse sources and alert those affected. Aggregation tool Flood Feeder, for example, puts together a feed of flood data from warnings, alerts, mobile phone mast locations and transport routes. Citizen Flood Journalism connects people tweeting from flood-affected areas, who can then contribute photographs and descriptions. Other apps allow users to predefine alerts for events – such as a power loss, a burst river bank or flooded roads and paths – and receive geographically-specific alerts by app or SMS. Other platforms use data to provide and co-ordinate assistance.

FloodRelief, for example, connects volunteers, local authorities and emergency services with those worst affected. Fludbud finds Twitter users who are near flood affected areas – who have retained internet connectivity and know people in need of help – and tweets them information about nearby volunteers and other ways of finding help.

But apps for assistance reach much further than the immediate emergency. FoodTrade connects farmers and food businesses to mitigate the consequences of farmers producing reduced arable and livestock yields. It upgrades these connections regularly to provide near-live information for farmers to find alternative routes to market and retailers to access alternative producers. As the developer points out, the app is just as useful for flood victims in the UK as it is for subsistence farmers in Costa Rica. Looking to the future, the Oxford Flood Network shows how data can mitigate the effects of flooding, with plans to build networked flood sensors under floorboards, bridges and structures around the city, from which to gather real-time data and a better idea of the anatomy of a flooding disaster.

Source: UK Flood Help February 2014

Tennison says a number of local authorities outsource website and digital technology from suppliers which can not change their systems rapidly or at a low enough cost to publish datasets. She also says some councils are facing legal barriers where their procurement contracts may have clauses which state that any data put into the outsourced system belongs to the supplier – who can then choose what to do with that data. While central government is trying to prevent this from happening in any future procurement contracts, by introducing standard transparency clauses, this may be preventing organisations from acting right now.

Leeds Data Mill project

One local authority set to make good progress is Leeds City Council. The city has just launched its independent Data Mill programme, in which open data files can be uploaded to a data management system and accessed by others.

The council backs Leeds Data Mill and has agreed to upload its datasets, but the platform will also host data from private organisations, including First Direct Arena and the West Yorkshire Metro. These organisations will join art galleries, charities, universities, travel providers and the health sector as the platform progresses.

Mark Barrett, open data lead at the Leeds Data Mill, says he set it up because he was "fascinated about data that could change people’s lives – data on a local level, rather than a national level". Before the launch of the Leeds Data Mill on 8 March 2014, the city council was already beginning to make data available, but Barrett says there were only 15-20 datasets available. But by the time of its launch, the Leeds Data Mill had 50 datasets in total – 30 of which were brand new – which included five datasets from the private sector. "In three months that was quite an achievement," he says. "We’re just getting going, there’s a huge appetite to turn Leeds into an open data city. "We’ll be able to understand our city better. Often, people have a hunch about what their city is but, until you have the data, you can’t prove anything."

The future of local government service provision

Barrett says city services often overlap, and it is not until the datasets are made available that patterns begin to emerge. Dylan Roberts, CIO of Leeds City Council, believes data and information is a "key element of success for Leeds as a city, not just the city council". But while Roberts says Leeds council chief executive Tom Riordan believes in the power of data, other business leaders across the council and the city still need some convincing. But the effects of austerity drives in recent years require no such leap of faith. Roberts says open data will help the council as it faces ever-increasing budget cuts.

"From our point of view as a council, we recognise it’s going to be impossible to deliver a full range of services as before," he says. "It’s more about delivering outcomes rather than services, and to do that through new models.” He says that, by empowering the private sector, entrepreneurs and even citizen 'bedroom coders' with data, the council will have a full range of stakeholders who will hopefully develop services which may replace what the council had provided previously. Roberts calls this “civic enterprise", adding: "The private sector becoming a bit more civic and a public sector becoming a bit more commercial."

Hackathon day for Leeds data

Barrett says the Leeds Data Mill will set challenges and ask the community to answer a data-driven question. The answer could be developed during a weekend hackathon, or participants could be commissioned to investigate a trend in more detail. He hopes startup technology companies will launch from making this data available.

Leeds City Council is set to open an innovation centre to encourage people to work on some of the city’s problems through hack days and entrepreneurial opportunities. A different approach to council procurement is needed, according to Roberts. He says hacking events could result in the council and developer co-producing a solution. "Fundamentally the idea for me is we’re providing the ground for something co-produced, almost like in-house development," he says. Off the back of a hack day, the council wouldn’t know what the results are going to be, so prototypes are important to find high-level problems early on, says Roberts.

It is still early days, but if the Leeds Data Mill manages to gain traction in both the public and private sector, it could prove a rich bank of information which could benefit the lives of citizens. This and Flood Hack prove the need to liberate data, but there are costs associated with the publication and maintenance of datasets. Local government IT directors also need to make sure they can access the data, especially if it is hosted by a supplier, who may not be able to provide the data in an easy and cost effective way.

Read more on Business intelligence and analytics