Using open data in local government

Developers are using open data from local government to create new, innovate public services. Cliff Saran reports.

In May 2009, Irving Wladawsky-Berger, visiting lecturer at MIT’s Sloan School of Management and Engineering Systems Division, wrote a blog that discussed the opportunities of smart cities. Wladawsky-Berger, who previously worked on IBM’s smart cities programme, noted: “Given our ability to now embed computational power into just about any physical object we want to measure and control, the digital and physical infrastructures of cities are converging.

We can now better manage critical urban infrastructures, including traffic congestion, energy usage, public transportation, and water distribution.”

Wladawsky-Berger posted the blog after a conference on smart cities held at Imperial College in London. “Just about anything we care about can be sensed and measured – any person, any object, any process or any service, can become digitally aware and networked,” he said.

Government could do more for open by default data

“We believe that there should be a public presumption (or formal duty on public sector organisations) in favour of open publication by default, with restrictions limited to personal data sets. However, we are concerned that while the publication of multiple CSV/Excel files may be welcomed by the technically adept, the format, without contextual information, may prove inaccessible to a lay user. Therefore, we believe the government could do more to develop open by default data standards that reference the importance of context.

“The government could provide additional incentives and lead by example. As a local authority we produce over 200 separate data sets in prescriptive, sometimes exclusive, formats for government departments annually. If the government were to commit to downloading the performance data they require direct, it would improve efficiency and enable us to manage our resources more effectively.

“An independent body (such as the Information Commissioner) with enhanced powers to challenge decisions not to publish data may assist in increasing the amount and range of data that is openly published. We believe such a body should also have a role in supporting/
enforcing accessible standards, including formats and presentation in the publication of data.”
Roger Hampson, chief executive, Redbridge Council.

The idea of smart cities relies on data and data collection to simplify and improve the lives of individuals and to manage public services more efficiently. The data can be used at a city level, a regional level or country-wide. It can be organised into so-called data stores, to enable officials and interested third parties to create new and innovative applications.

Low cost of entry

“If the state makes the data it collects available royalty-free, then a developer does not have cost to bear at the point of entry,” says Emer Coleman, deputy director of digital engagement at the Government Digital Service. Coleman, who previously worked at the London Datastore, says: “One of the first Barclays Bike apps was built by someone who was 19 or 20.” There is a very low cost of entry and it does not matter whether it is an individual developer or the likes of Google or Microsoft. “The nature of the apps market is that the best ones will rise to the top,” she says. As the apps developer starts making money it will begin paying taxes, providing a revenue stream for government, she explains.

Coleman was one of the speakers at the Open-data Cities Conference held in Brighton on 20 April.

Writing on the Computer Weekly Enterprise blog, Greg Hadfield, a former Fleet Street journalist and internet entrepreneur, organiser of the event, said: “The more data that is released – without strings attached, in machine-readable and non-proprietary ‘open’ formats – the more likely it is that businesses and developers will use it to build the applications and services that world-class cities need.”

Tying local information

Ordnance Survey runs open data master classes around the country. Bournemouth abandoned vehicles is one of the applications in local government using its data. Ordnance Survey provides various tools for developers, such as the Open Space web service and Web Map, a wizard-like tool to create maps in Javascript.

Ian Holt, developer programme manager at Ordnance Survey, is also encouraging people to use open data. He says, “What is really interesting is that before open data it was quite a process for people to work with Ordnance Survey.”

Tying geographical data with locale-specific information can lead to innovative applications that make it easier for managers to visualise trends. “We are seeing people take data in charts and look at patterns in other data sets. People can style data and modify it in ways we would have never envisaged.”

One example is iCoast, a map of the Dorset coast that uses Ordnance Survey charts in a way that Ordnance Survey would not have done itself. Holt adds, “We are also seeing our data in Facebook.”

Local data groups

Mayor Boris Johnson launched London’s open data project, called the London Datastore, in January 2010. It was created by the Greater London Authority (GLA) as a way to release London’s data. “We want citizens to be able to access the data that the GLA and other public sector organisations hold, and to use that data however they see fit – free of charge,” the London Datastore states on its website.

The project aims to make large amounts of London’s public data freely available through the London Datastore website. It comes after the government announced it would make national data sets available via the internet, following the project in the US.

Victoria Borwick, chair of the London Assembly Health and Public Services Committee, recently wrote on her blog about how open data could be used to track fuel poverty: “Existing open public data sets can be used to identify wards that are likely to contain high proportions of households at risk of fuel poverty, and programmes could be targeted to provide support to households in those areas. This provides information at a sufficient level of detail to help area-based schemes like the RE:NEW programme to prioritise their time and resources to help those most in need.”

Borwick’s team worked with the GLA’s Intelligence Unit to develop a prototype mapping tool to identify areas in London containing large proportions of households at risk of fuel poverty, using a combination of publicly available data sets. She added: “We want the GLA to further develop and refine this tool so it can be used by the GLA, local authorities and energy companies to identify priority areas for support to tackle fuel poverty.”

Engaging the public

The Local Public Data Panel is another group championing the release of local public data and information sharing. The Panel is looking at the role data transparency may play in increasing efficiency, extending accountability and engaging the public across the local government sector.

Roger Hampson, chief executive of Redbridge Council, has served as one of the Local Public Data Panel’s experts since its inception in January 2010.

Redbridge began working on open data in late 2010 when it established a small in-house team to develop its own open data application called DataShare. Hampson says: “Our core aim is to tap into the sea-change in the approach of governments worldwide to public data.” The council’s aims for open data include meeting government expectations to publish data and increasing openness and clarity, which may help to increase public trust. Other benefits include saving time and money by streamlining/replacing multiple internal data publishing systems.

According to Hampson, very little work was required to make data public. “We have a long-standing strategic commitment to publishing all (legally permissible) data with a view to extending accountability and public engagement. We also believe that making public data very visible is, paradoxically perhaps, very useful to bureaucrats and politicians.” The application has now been populated with data sets covering all service areas, including:

  • Staffing and salary information;
  • Service expenditure over £500;
  • Aggregated returns to government departments (200+);
  • Previous FOI requests and de-personalised answers;
  • Information in respect of frequently asked FOI requests;
  • The data sets, extracts and reports from some council back end systems;
  • Geospatial information drawn from existing applications such as planning and ‘report it’;
  • Requests for data sets from members of the public;
  • Innovative visualisations/apps produced from the raw data by external developers.

Hampson adds: “We designed DataShare as a ‘plug and play’ application with a schema flexible enough to host most, if not all, of the data publishable by local authorities. We have recently entered into a partnership arrangement with CIPFA to promote the DataShare tool and extend take-up across the sector.”

As well as working with CIPFA, Redbridge has participated in a ‘hack’ morning, during which developers downloaded data from the site to create accessible visualisations on FOI and the geographical spread of expenditure over £500.

Opening up data

Lambeth is another example of a local authority taking major strides with its use of open data. The process for opening up data at Lambeth begins with the council’s data team, which is responsible for releasing data openly and generally takes the lead in deciding which data to release. The council has conducted a benchmarking exercise against other local authorities and mapped out a range of data that it could potentially publish. This was reviewed by the data team, which drew up an open data schedule.

Members of the public can also request data for release by contacting the council by email.
The council’s data team then works with service managers to identify data owners and processes the data into the relevant open format (csv, kml). Identified data owners are then responsible for ensuring the data team is aware of any updates or additions to the data. The data team will aggregate the data if necessary to ensure data protection; the communications team will also advise the service manager with responsibility for the data if needed.

“We are not really aware of the extent to which software developers are engaging with the council’s expenditure data, although we anticipate this being low,” reveals Matthew Skinner, policy, equalities and performance officer at Lambeth. However, he says the expenditure data has been reused by Openly Local, which developed a ‘spending dashboard’, to demonstrate the money that has been spent to date by local authorities such as Lambeth.

The Openly Local site pulls a range of open data from across local government. It can be accessed in xml, json or rdf formats via an application programming interface.
The Great British Toilet map project also made use of the toilet location data that Lambeth released to its open data pages.

Meeting data requests

Tom Steinberg, founder and director of mySociety – a charitable project involved in open data – was one of the speakers at the Open-data Cities Conference. He says: “While government has always produced information, the value that can be realised using data with the internet is far more valuable, such as postcodes that were invented to help deliver letters. Now postcodes help run businesses.”

Steinberg believes that while there has been a lot of talk about open data as a commodity, this view misses the focus. “People need certain data to do their job, or hold the government to account,” he adds.

For Steinberg, hacking days and data stores miss the open data point, because these rely on the council being organised around which data it publishes. Generally, he says, councils do not publish the data people want, which leads to a lack of adoption. Councils have to get used to responding to data requests. This means each council’s helpdesk must be able to identify a call as a request for data. There needs to be someone in the council who takes responsibility of the data request. The problem for Steinberg is that back office systems used in local government are not very good at making data available externally.

Leigh Dodds, chief technology officer at Kasabi, takes the concept of openness of public data a step further. Speaking at the Open-data Cities Conference, he said: “Any data needs to engage with a wider community. It helps if everything in the council can be identified as a url. People can then tweet it and discuss it online.”

And while it is not local government, the BBC digital archive is organised this way. Bill Thompson, head of partnership development, archive development at the BBC, says: “Opening up our data makes it possible to complement other open data initiatives and create a digital public space.”

This could lead to new services such as being able to find out what programme was being broadcast when an individual was born, find buildings from the archive that do not exist any more, even compare how politicians change their perspective over time.

Read more on Web development