Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens

Is the campaign won? What do you think?

Choices by anemoneprojectors.

Choices. Photo by anemoneprojectors on Flickr. Some rights reserved
Simon Rogers writes at the Guardian about the release of a snapshot of the Treasury’s COINS (Combined Online Information System):

“known universally as Coins [it] is the most detailed record of public spending imaginable. Some 24m individual spending items in a CSV file of 120GB presents a unique picture of how the government does its business.”

Although that’s because it was presented in UTF-16, which would be great if it were encoding Mandarin (in-joke?), but which actually meant that every second byte was blank – so the Guardian’s developers simply did a quick conversion to UTF-8, halving the size at a keystroke.

He notes:

“This is a different kind of database. It shows how the government actually works; the millions of tiny items that make up the billions of public expenditure every year. It could well be the government’s largest database: if you know of anything of equivalent size and complexity let us know – we can’t come up with anything.”

And then he comments:

“It was only 2006 that the Guardian launched the Free Our Data campaign to push for the government to release public data that we’ve paid for but was previously hidden behind paywalls or official secrecy. Now, that battle is won.”

That’s an interesting one. For me, Coins isn’t actually the marker for the point where we can hang up our campaigning clothes. The Ordnance Survey data release was a huge step. But actually, for me the point at which I’ll feel we’ve really reached the parts that we want to reach is when the Environment Agency makes its flood map data available for free commercial re-use. (I haven’t asked Mike Cross. Perhaps he’ll pitch in.)

But what do you think? It’s your campaign too. Is everything that needs to be done, done? Or is there more to be done, and if so, what?

18 Responses to “Is the campaign won? What do you think?”

  1. Tim Says:

    It’s been a successful campaign by any measure. Well done.

  2. Steve Says:

    “and if so, what?”

    I guess the photo was chosen for other reasons, but it answers your question…”Public Bridleway…footpath…byway” Just where do I get this data from? The OS won’t release as they don’t own the data, the LA’s won’t release as they have been mapped onto an OS map, so are a derived work. The OS have said they won’t worry about the derived work part, so its back to the LA’s, but they are too spread out to provide a data source.

    Only the government can solve this one.

  3. William Allbrook Says:

    Yes, a fantastic job but it’s not done until the derived data issue is sorted.

  4. Robert Whittaker Says:

    “The OS have said they won’t worry about the derived work part”

    @Steve: Do you have an official source that can be quoted for that?

  5. Paul Says:

    @Robert as you and I have been discussing over on the OS blog public rights of way is a tricky one in terms of IP and ownership. I’ve asked the legal team to give me a definitive answer about any derived data issues and I’ll let you know as soon as I get it.

  6. Adrian Says:

    Until the “address IP war” is resolved with RM, OS and LAs all claiming, IP over the national address data and restricting use, the war is not over. We are still in a position where £10Million is being spent to produce a one off address database for the 2011 census, joining all sources, which will be not used for anything else.

  7. Richard Says:

    Also OS is GB only, we need to know where OSNI stands in all of this. The free Codepoint file is GB only.

  8. Brian Williams Says:

    I don’t believe that the 1:25000 data is freely available for walkers yet, is it?

    If I can’t extract a 1:25000 bitmap for my smartphone / gps using software that I may write myself, then the job isn’t finished. Can I?

  9. Brian Williams Says:

    Wow. Who wrote this piece? Was the question rhetorical, or a way to get the kiddies to talk amongst themselves?


  10. Charles Arthur Says:

    Hi Brian – it wasn’t rhetorical: it was intended to get people to talk about what things they wanted and how it was viewed.

    The point about PROW is a good one, particularly. So is OSNI.

    The 1:25K data (Brian) isn’t part of the free data; that’s a compromise that I think we have to accept for now. There is OpenStreetMap (which is quite good at 1:25K, especially as it now incorporates Street View at 1:10K).

    So I don’t think the campaign is entirely won. For my part, I’d like to see the Environment Agency’s flood map data made available: there’s a lot that can be done with that, potentially.

  11. Richard Says:

    Re OSNI, OS don’t seem to know and you e-mail OSNI and don’t get an answer

  12. CNM Says:

    They are probably making a lot of money out of the flood map data so they will be reluctant to give it away especially with the new budget and all.

  13. Micah B Says:

    Well it won’t be won till we can freely use Ordnance Survey Mastermap data. But there are also now some very achievable goals in our grasp that will be very worthwhile victories to build on those so far.
    (as well as ensuring one that seem achieved are consolidated )

    Lets encourage the release of the National Street Gazetteer and the National Land and Property Gazetteer There is a mess with overlapping claims on various IP they contain. But all by bodies controlled by the government, the Royal Mail for postcodes, Ordnance Survey for some locations, and local authorities for bulk of it. In fact there is a diminishing amount of IP that is not own by local authorities as they continue to build and refine it. Something they will continue to do for the sake of there own efficiency as well as obligation. Postcode might not be a problem so much now and could be stripped from it.
    I hope a case could be made that if the NLPG was free- reused that the feed back corrections would compensate lose of revenue alone. The more used and tested a data set is the more it will be accurate if there is an effective corrective feed back mechanism in place. But the case that I look to is for society as a whole. In fact a case cropped up on a Radio 4 programme where someone was refused credit because the Postcode Address File that credit agencies as well as many companies use did not distinguish between a couple flats in split house but the NLPG did. The comes from local authorities that responsible for street naming and numbering but it is too expensive to be used in addition by credit reference agency.

  14. Brian Williams Says:

    For my part, OpenStreetMap is useless for topographic purposes, i.e. walking the countryside.

    The topographic data will have been collected mostly with taxpayers’ money as the topography does not change a great deal (contours, rivers, PROW, etc), so I think there is a greater argument for that to be free than the regularly-surveyed data.

    Of course, paper maps are probably the real money spinner for the OS as joe public has to buy them.

    It seems very wrong to me, however, that you should have to pay for the same data twice, once on paper, once on your handheld, and when people feel that something is wrong, they tend to pirate.

    I’m not saying that I would, but I was making an analogy with entertainment media.

    George Lucas for example sells us “Star Wars” on VHS, Laserdisc, DVD, Bluray at some stage. He sold the epidsodes individually as they were released, then in boxed sets. The Original edition, slightly updated edition and special edition and all combinations thereto.

    As a walker, I have to pay for the electronic edition as well as the paper edition in all scales that I need. It seems the same to me.

  15. commercial property maintenance leeds Says:

    I’ve emailed OSNI to get clarification on a few points, they seem very lax in replying (two weeks now, 4 emails sent).

    I havent yet had a read reciept, so either they are binning my emails or just not checking them at all. Excellent service OSNI.

  16. Paul Says:

    I can see that alot of data useful to some has been released, but it’s certainly not case won – some areas are getting worse as it seems likely that the excellent Multimap Open API is going to be withdrawn in the near future.

    It’s not just that 1:25k isn’t available – 1:50k isn’t available either apart from with a tiny tile limit on Openspace.

  17. Richard Says:

    It looks as if the proposal for the National Gazatteer with free access to the public sector but high cost to the commercial sector means that the fight will need to gear up again

  18. senior homes Says:

    Also OS is GB only, we need to know where OSNI stands in all of this. The free Codepoint file is GB only.

Leave a Reply