This blog post comes to you from a train travelling through the morning fog towards London, where I’ll be attending the ODI Summit today. I’m there this afternoon to launch the work I’ve been doing on the state of open data, work that was commissioned by Omidyar Network earlier this year. You can access my report – “Open Data: Six stories about impact in the UK” – on the Omidyar website. And if you’re at the summit today, then come along to my talk at 15:50 in NFT2 to pick up a copy.

When Omidyar Network approached me to write this report, I was attracted to the project as a way to bookend the research work I started on open data with my study into its swift adoption as a policy in the UK and US. But I was aware that in the time I had spent away from the open data field much had changed. Here in the UK, the government’s commitment to open data had led to the establishment of the Open Data Institute – the hosts of today’s summit and a catalyst for mainstreaming ideas about open data that I had witnessed cradled by activist communities since the early 2000s. Internationally, the development of the Open Government Partnership had taken open data global. And grassroots networks like Open Knowledge, on whose board I served in its early years, were swelling at an almost frightening pace.

It’s not often you see ideas catch fire like that. But for all that the open data world had grown and changed, it became clear as I began my work for Omidyar that pinning down the effect that government open data and the intermediaries that transformed it were having on the wider world, on the lives of ordinary people or on the most serious issues that affect us today, was not an easy task. Through early conversations with generous thinkers like Rebecca Rumbul, Júlia Keserű and Tim Davies, I began to understand the various challenges inherent in studying the impact of open data with rigour. Impact comes in many forms, and at many different points along the open data value chain. Barriers to impact can be multiple and entrenched. The theory of change behind open data resists traditional impact measurement. And it may yet be too soon to see the policy’s true impact. At the International Open Data Conference, held this May in Ottawa, I presented these ideas and they were greeted with relieved recognition from many of the scholars and practitioners there.

My presentation at today’s ODI Summit is called “It takes a millions stories”. That’s because although we may be able to speak of open data’s potential in sweeping terms (and another major report Omidyar commissioned on open data, from Lateral Economics, did just that, predicting in 2014 that open data could achieve more than half of the G20’s 2% growth target over five years) the actual, tangible impact of open data may only ever be visible in a million tiny fragments. My report deals in six of those fragments – or five, depending on your point of view, since I’ve included one story about closed data and impact, as a nod to the counterfactual.

For fragments, they’re pretty awesome. TfL is saving itself (and the farepayer, and the taxpayer) millions of pounds outsourcing app development through releasing all its data openly. It turn, those apps are saving London’s transport users tens of millions of pounds in monetised time savings. The monetised time savings approach (not new to this report, but developed by Deloitte using data from TfL and the Department for Transport) can also be deployed to put a pounds and pence figure on how mySociety are lowering the cost for civil society groups and journalists to engage with the Parliamentary process. And TheyWorkForYou.com, mySociety’s accessible version of the records of Parliamentary proceedings – itself practically the seed of the entire open data movement in the UK – may actually be contributing to a long term, positive change in the way Parliamentarians vote. You can read the full report on all six case studies here.

The theme of this year’s ODI Summit is Generation Open, and I’m looking forward to hearing more about the research ODI has been doing into the attitudes of the next generation to open data. My hunch is that it will be close to what Chris Taggart said to me when I was conducting research into the role OpenCorporates had in the campaign to purge the UK of anonymous shell companies, a campaign that succeeded in getting David Cameron to commit to a public register of the beneficial owners of companies in the UK. When I asked him the best way to talk about open data’s impact, he said:

“It’s a bit like [asking for] the evidence basis for a justice system. We need a justice system because that’s the society we want, one with a justice system, with the rule of law …. Let’s stop trying to argue for open data, and make [opponents] argue for closed data, right?

These are fighting words, and they are ones with which I have a lot of sympathy. Because for all Generation Open intuitively understands open as a default and not a challenge, there are those in power who do not share their views. Witness the selling-off of the PAF alongside Royal Mail. Or the threat of privatisation to the UK’s trading funds and current gatekeepers of our national information infrastructure: HM Land Registry, the Met Office, Ordnance Survey and Companies House. Or the fact that this government currently has the Freedom of Information Act under review.

In a piece I’ve written for openDemocracy today, I argue that the open data community now faces two challenges:

Firstly, and most urgently, it must win over those at HM Treasury who have yet to accept the economic arguments for putting public data in public hands, in time to rescue the nation’s information infrastructure from privatisation…But the open data community must also recognise the need to go further than the bottom line, to stop masking aspirations for a more transparent and accountable society driven by open government data in the language of economics, and to adopt a more rights-based agenda.

It’s a challenge I plan to put to my audience later today, and it’s one I use to close off my report. Open data is not just an economic policy, it is a democratic imperative. It’s time for the open data movement to stop playing nice.

As part of the work I’m doing tracking the impact of open data in the UK, I’m asking politicians, civil servants, journalists, activists, campaigners and community volunteers who use TheyWorkForYou.com to complete a short survey. I’m trying to understand the value TheyWorkForYou.com delivers to the people working inside Westminster, Holyrood and Stormont and to those on the outside seeking to report on or influence their activities.

The survey should take no more than ten minutes to complete. I’d like to be able to attribute your comments in my final (public) report, but there is also an option to contribute unattributed. If you want to help out, please access the survey here.

Please note that I am not affiliated with TheyWorkForYou.com or mySociety, although I did do some copy-writing work for them about three years ago. You can find out more about me and the report I’m writing here.

This is an extended form of the short presentation I made at the 3rd International Open Data Conference today.

I’m working with Omidyar Network to document – through a selection of case studies mainly based in the UK – the impact so far of government open data policies. There are a lot of people thinking about open data impact right now, not least my companions on this panel, and I wanted to take this opportunity to briefly lay out my perspective on the issue, which as I mentioned already is grounded for the most part in the developed world.

When the UK and US governments adopted the open data policies that have led to them being leaders in this field, they did so on a prospectus of potential, and not proof. Open data policies promise to deliver economic efficiencies, governance gains, and social and environmental goods for what in government spending terms is a relatively low initial investment. They promise to keep bureaucracies one step ahead of the internet’s disintermediating power. Sold as such, it’s easy to see why governments in the midst of a global financial crisis and technological revolution bought into open data.

Great expectations

Our expectations have been set high, perhaps too high, in terms of what open data will deliver. Indeed, searching for tangible impacts at scale only three or four years after implementing a long-term policy change such as opening government data, we may yet be disappointed. And it would be a great shame if that disappointment led to disengagement with open data policy. Thankfully there are some front-runners, some stories we can tell about where open data is making a difference right now, and I hope these stories are enough to keep us on the right track, towards a world of open by default.

But we also need to be realistic about how much other factors, factors beyond the availability of information through open government data, affect the impacts we want to see. I want to show you this graph devised by Rufus Pollock of Open Knowledge:

Seeing this graph was a lightbulb moment for me. At a basic level, it illustrates the vast range of issues upon which we expect open data to have some effect. But it’s also a reminder that, as Rufus puts it, open data is a complement, not a substitute, for institutional change. We can’t expect unicorns and rainbows just by creating another CKAN instance. Most of what open data will achieve, it will achieve in partnership with some other kind of hard work.

Elusive evidence

Back to those good news stories. A study by Deloitte for the UK’s Department of Business estimated that by opening up their transport data, London’s transit authority netted efficiency savings of £58 million for London’s transport users in 2012. As the authors of that study observed, that is a considerable percentage of the projected annual savings that were used to justify the creation of an entirely new and very expensive high speed rail link in another part of the UK.

Playing around with numbers is one of the nicest ways to talk about impact, and in countries like the UK, where the civil service has actually created standard estimates for what an hour of my time is worth to me, it’s relatively easy to do. But talking about the wider social or political impacts that flow from the release of a particular data set is a lot trickier.

Even if you start trying to equate a country’s open data policy with shifts in its ranking on this or that social or political global performance indicator (and those sorts of indices bring their own issues with them), you’ll find causality hard to prove, not least because such indices are extremely high level, and the open data value chain is very long indeed. And you can add to this the problem that most studies designed to measure impact want their protagonists to have a set of goals they’re working towards, and that open data, by contrast, says “we’re releasing this to the world precisely because we believe we alone don’t know all the great things to do with it”.

Story-telling

Hard problems like this are of course great for academics. But I’m not an academic. My background is in journalism and activism. Essentially, I write stories. And right now, when it comes to open data impact, I don’t think that completely disqualifies me from the job at hand.

The best example we have so far of benchmarking open data impact is the Worldwide Web Foundation’s Open Data Barometer. And the way they measure impact is – they see how often open data is mentioned in the press and online. This is, in my mind, pretty much equivalent to measuring how good a story it is.

My stories don’t need fairytale castles or knights in shining armour, but they do need to resonate with someone who is unlikely to ever attend a hack day, or know what a Hadoop cluster is. So if you think you are sitting on such a story, or you know someone else who might be, I’d urge you to come and find me over the next couple of days.

Thank you very much.

Great slide by @barefoot_techie at #IODC15 Popular sentence during the past few days 😉 pic.twitter.com/7HinYXRHvs

— Silvana Fumega (@SilvanavF) May 28, 2015