Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens

Archive for June, 2009

Michael Cross: setting data free is an easy promise when in opposition – so would a Tory government do it?

Friday, June 26th, 2009

Michael Cross, co-founder of this campaign, has an article at the Guardian’s Comment Is Free site on the Conservative pledges on data made on Thursday by David Cameron. Of note:

The three-year-old Free Our Data campaign – founded by myself and the Guardian’s technology editor Charles Arthur – will welcome Cameron’s re-stated promise to publish every item of government spending over £25,000 and raw data to allow communities to build their own crime maps and councils’ performance data in a standard format.

We will cheer most loudly at the plan to create a new right to data and proactively to identify the 20 most useful data sets on public services and make them available for web mash-ups.

But, he points out, there are warnings to be heeded.

To judge by Cameron’s speech, which makes no mention of the government’s single largest data business [Ordnance Survey], the Conservatives share this aversion to reform. The suspicion must be that the Tory solution is to try and sell off the mapping agency lock stock and barrel. Yet locational information is an essential component of nearly every public data set. To commercialise its supply would be to move in the very opposite direction of setting our data free.

It certainly is important for the Conservatives to set out clearly what their intention is with regard to OS before the election. A manifesto commitment not to sell it off would be a good idea.

Read the whole article for the wider points. Steven Feldman likes it – and adds

My one question is if the treasury are unable or unwilling to go down the centrally funded route what would you prefer – privatisation or trying to get the best out of the current model. I know which one I would choose.

David Cameron gives speech suggesting “setting data free”

Thursday, June 25th, 2009

David Cameron has given a keynote speech which continues to edge the Conservative party towards something that might look like the glimmer of the beginnings of the outline of the rough shape of a manifesto.

Part of it was to do with what he called “Setting data free”. See what you make of it.

In Britain today, there are over 100,000 public bodies producing a huge amount of information.

This ranges from school league tables to train timetables; from health outcomes to public sector job vacancies. Most of this information is kept locked up by the state. And what is published is mostly released in formats that mean the information can’t be searched or used with other applications, like online maps. his stands in the way of accountability.


… what about patient outcomes in the NHS? Some of the most important information you’ll ever need to know, how long your Dad will survive if he gets cancer, your chances of a good life if you have a stroke, all this is out of your hands.

Now, again, imagine if this information was in your hands. You’d be able to compare your local hospital with others, and do something about it if it wasn’t good enough. Choose another hospital. Voice your complaint to a patient group. Make change happen.

All this data which would help people in this country hold the powerful to account – it’s all locked away in some vault. And it’s only getting worse.

We’re going to set this data free. In the first year of the next Conservative Government, we will find the most useful information in twenty different areas ranging from information about the NHS to information about schools and road traffic and publish it so people can use it.

This information will be published proactively and regularly – and in a standardised format so that it can be ‘mashed up’ and interacted with.

What’s more, because there is no complete list that can tell us exactly what data the government collects, we will create a new ‘right to data’ so that further datasets can be requested by the public.

By harnessing the wisdom of the crowd, we can find out what information individuals think will be important in holding the state to account.

And to avoid bureaucrats blocking these requests, we will introduce a rule that any request will be successful unless it can be proved that it would lead to overwhelming costs or demonstrable personal privacy or national security concerns.

If we are serious about helping people exert more power over the state, we need to give them the information to do it. And as part of that process, we will review the role of the Information Commissioner to make sure that it is designed to maximise political accountability in our country.

The suggestion that councils will be obliged to publish data in a common format is one that Cameron has made before; and it’s something that Adrian Short’s mashthestate has been pushing (and successfully) without any political party.

The idea though of the “right to data” and the assumption that data should be available is interesting. Allied to Sir Tim Berners-Lee suggesting ways to get the data out there, it looks like all the political parties think that making data free – in the sense of untrammelled, if not yet in the sense of not-charged – is an idea whose time has come.

Using Gapminder to compare the countries studied by OS: how did they choose?

Wednesday, June 24th, 2009

See this screenshot on Flickr

Interesting to look at the countries which Ordnance Survey studied for its, er, study. Note that the study – which one would have hoped might be at least exhaustive, since it’s a one-off chance to really show things – excluded South Africa because it doesn’t have a comparable GDP per capita to the UK. Apparently comparable GDP was a specific measure used to decide who to examine.

Well, Gapminder offers the chance to analyse countries’ GDP per capita.

OK. So OS looked at Australia, Canada, France, Netherlands, Norway, New Zealand, Sweden, and the US – which, as Gapminder shows, do indeed have comparable GDP per capita. (This link may give you the graph – though you might have to click on the “map” link, and then go back to the “graph” link and play the runthrough.

But wait, what’s this? If you look at the graph, there are all sorts of other countries in there with comparable GDP/capita.

Even between New Zealand (the lowest) and Norway (the highest – didn’t expect that, eh?) there are loads of countries.

To wit: Bahrain, United Arab Emirates, Cyprus; Japan, Hong Kong. Monaco, Isle of Man, New Caledonia; Ireland, Denmark, Finland, Greenland, Sweden; Germany, Austria, Spain, Switzerland.

Anyone want to suggest any reasons for ignoring those? And what the business models are for their mapping agencies?

And it doesn’t make much sense if you compare GDP/capita against land area either.

Here’s the picture. Go analyse yourself – please. I’m too puzzled.

Ordnance Survey provides redacted version of its study into its financial organisation

Wednesday, June 24th, 2009

Ordnance Survey responded to our FOI request for publication of its study into itself and the best financial organisation for itself.

And in these times when MPs’ expenses are redacted, of course OS isn’t going to let us see everything.

That’s why the document, which I’ve uploaded as a PDF (though it’s originally a TIFF – apparently a scan of the paper document once the black marker pen had been wielded), is full of lacunae.

After some struggles (apologies) it’s now available as a PDF (3.7MB).

Have a look for yourself and see what strikes you as interesting.

This isn’t the final version of what we’ll get, however. In its response to me, OS says

Please see attached a redacted version of the International Comparison of Geographical Information Trading Models – Study report (ref: 71171). The report was commissioned by the then Parliamentary Under Secretary of State for Communities and Local Government, Iain Wright MP, formed part of an input to the Trading Funds Assessment undertaken by the Shareholder Executive on behalf of HM Treasury and the Department for Business Enterprise and Regulatory Reform (now Business, Innovation and Skills).

As you are aware this is an interim report while we wait for approval from the countries/agencies, who took part in the study, to release information related to them. The extension date for the next version of the report is 23 July 2009.

We’ll compare the two, of course. But it’s likely there will still be stuff cut out because it’s too sensitive for us poor souls.

OS first response to FOI questions: who did it talk to, and who helped? (Updated)

Tuesday, June 23rd, 2009

Ordnance Survey responded today to our FOI request. The redacted financial study is expected later today, but first here are the organisations that it talked to in deciding whose model to adopt. Or not adopt.

Here’s the text of the FOI response.

The report about which these question were raised, was commissioned by the then Parliamentary Under Secretary of State for Communities and Local Government, Iain Wright MP, and formed part of an input to the Trading Funds Assessment undertaken by the Shareholder Executive and HM Treasury.

1. Who or what was the “outside help”?

With regard to the International Comparison of Geographical Information Trading Models Study, outside help was provided by senior officials of those Institutions contacted.

In the case of the United States of America, as senior officials of the United States Geological Survey (USGS) were unavailable, Mr. David Cowen, Distinguished Professor Emeritus at the University of South Carolina, kindly provided us with an in-depth overview of the state of public sector GI data in the United States, including USGS. Mr Cowan is a former chair of the Mapping Science Committee of the United States National Research Council and is chair of the National Research Council’s Committee for the study of Land Parcel Databases.

The document was also reviewed by an internationally recognised expert in Geographical Information and National Mapping who agreed with the analysis and conclusions.

2. Which “equivalent organisations” were examined for the study?

There are no wholly equivalent organisations to Ordnance Survey, given its range of scales of mapping and other activities. Hence a representative sample of eight National Mapping Agencies or their closest equivalents were examined during the study. The sample included organisations with a wide range of data pricing policies: free, partial cost recovery (recovery of data dissemination costs), total cost recovery (recovery of data collection plus dissemination costs) and market price (cost recovery plus trading margin).

The overseas examples studied were:

Australia PSMA (Public Sector Mapping Agencies)
Canada Natural Resources Canada
France: IGN (Institut Geographique National)
Netherlands Kadaster
Norway Statens Kartverk
New Zealand LINZ (Land Information New Zealand)
Sweden Landmäteriet
United States USGS (United States Geological Survey) – via Mr David Cowan.

3. Which agencies did OS examine for the study?

IGN, Kadaster, Statens Kartverk and Landmäteriet have Agency status within their respective Governments.

4. Which agencies did the “outside help” examine for the study?

As indicated above, the outside help for the International Comparisons Study was provided by senior officials from the National Mapping Agencies examined in the Study, together with Mr Cowan and the internationally recognised expert.

And finally…

Please note that your enquiry has been processed to Freedom of Information guidelines. To the extent that the public interest (section 17) applies, we have determined that in all the circumstances of the case, the public interest has been met with the full provision of all information in this instance.

Update: Steven Feldman, who has been consulting for OS on how to get wider adoption of its OpenSpace API on its GeoVation initiative. Quoth Feldman (in the comments below): “We expect to go public on what we have planned for GeoVation in the next couple of weeks, watch my blog or mail me if you want to get a mail from me when we kick off.” (We apologise for the earlier error.)

, says he is not the “internationally recognised expert” mentioned by OS. We were going to ask but he beat us to it. The search goes on…

Reorganising this blog: different category suggestions

Friday, June 19th, 2009

Speaking to someone this morning about the Free Our Data campaign, I found that I was constantly saying “Oh, we wrote about X on the blog…” (where X could be the Postcode Address File, or the proposal to charge for changes in the Land Registry, or whatever) – but then realised it’s very hard to find anything by that label here. Search is nice, but it’s not enough.

Therefore I’m proposing to update the “categories” here into a number of new ones, principally with the names of organisations – eg Ordnance Survey, Post Office, Hydrographic Office, Land Registry – that would be affected or included in the post.

Your suggestions welcome for how, for example, we should deal with the “Cambridge report” or the Trading Funds review. And for any other organisational change to the categorisation on this blog for posts that would make your use of it easier.