What’s changing in the open data ecosystem?

Last weekend was the 3rd Open Data Camp, in the great venue of the Bristol Watershed. Across the many sessions and discussions over the 2 days, there were some clear stories of what’s changing in the open data ecosystem, and some clear frustrations about what’s still needed.

The open data centre of gravity in government appears to be shifting towards Defra, at least to us observers outside government. A combination of top-level support fromministers and senior leadership is helping drive a big ramp-up in activity and data publication. At Open Data Camp there was a big turn-out from Defra and Environment Agency (although it was a bit of a home game for the Environment Agency with their Bristol HQ), and lots of discussion around data such as Lidar. With many of the current good examples of data use coming out of Defra, Environment Agency et al, next month’s Defra Open Data Market event will be a good event to take stock of how far we’ve come in opening up useful data.

There’s still a massive need for improvement in the “find, understand, use” part of the open data ecosystem. Data.gov.uk and other local open data systems are still essentially simple catalogues with only basic search tools – and have not really evolved in user-terms since open data catalogues such as our own Data4nr.net appeared in 2005. There’s little linkage between these data catalogues and “how the data has been used”, and little-to-no linkage with help on “how do I use this?”. There are some bright spots out there: Data USA  and the recently launched Data campfire are based around telling data stories, Nomis’s help forums are a truly useful source of expert help, and the Stack Exchange Open Data forum is interesting but needs more support and momentum (and perhaps a UK-specific version). I understand GDS are reviewing data.gov.uk, and it would also be good to see ONS impact in this area – the National Statistician role includes data dissemination across government, not just ONS data. If we’re serious about continuing to help users use data to improve services and businesses, it’s time we got serious about improving this part of the open data ecosystem.

 

It’s time to move on from asking “is open data valuable”? There are 100s of examples of open data proving its worth – from Census data (“2011 census benefits were £490 million each year”, Ian Cope ONS) to the Index of Multiple Deprivation being used to target upwards of £1billion resources per year to open transport APIs powering consumer travel apps to recent Lidar use (more on that below). Open data demonstrably provides value. Of course that doesn’t mean that every open data set is valuable – you can look at the usage statistics for data.gov.uk to see some of the less useful candidates (the CSV download athttps://data.gov.uk/data/site-usage/dataset shows all datasets, and there’s a very long tail) – but can we please stop asking the “is open data valuable?” question.

Data use gets creative. For me the highlight session at Open Data Camp was John Murray’s step-by-step run through from raw Lidar height data to filtered building outlines.  The task that the Environment Agency set our Data Advisory Group in the first meeting was to prioritise which of their datasets they should release first. Lidar was absolute top of our list, and in meetings with the Lidar data team we listed roughly 50 uses for the dataset that helped make a bullet-proof case for publishing as open data – many of which we’re already seeing (although we missed the Roman roads … ). There’s a lesson here about the value of open data – although the Environment Agency EA no longer receives licensing fees from the (now) open data Lidar dataset, the return-on-investment to the Agency’s task and public realm is far more significant.

Open Data Camp was a great community-building event, very much down to the organisers for their hard work in putting it together and bringing in so many of the people doing great stuff in this field. I’m looking forward to the next.

Tom Smith is Chief Executive of OCSI and chair of the Environment Agency Data Advisory Group. @_datasmith and tom.smith@ocsi.co.uk.

Originally featured on Open Data Camp

---

Featured posts