Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens


Archive for 2009

International man/woman of mystery is: international; from an NMA

Monday, August 3rd, 2009

The OS has responded to our latest FOI request to try to establish more details about its international expert. And what do you know, they really are international.

And they used to work for a national mapping association (but not OS.)

Keep your ideas coming….

1) Is the “internationally recognised expert (a) British or (b) a foreign national?
A foreign national.

2) Is he or she a (a) currently employed in a UK academic institution, or (b) employed in the UK, or (c) employed in a UK government department, or (d) a UK government agency or trading fund.
No.

3) If so, which institution, business/fund or arm of Government?
Not applicable.

4) Has the person retired from any of those listed in (2)?
Not applicable.

5) Has the person ever been employed by the Ordnance Survey and if so on what basis (eg full-time, continuous part-time, etc)?
No.

6) Has the person ever been employed by a foreign National Mapping Association, and if so on what basis (eg full-time, continuous part-time, etc)?
Yes; full-time.

(This would be why Ed Parsons – well, we had to ask – and Dave Lovell, executive director of Eurogeographics, have turned out not to be the person. Only 5,750m people to go then…)

..but we’re just as quick: more questions re the international man (or woman) of mystery

Friday, July 31st, 2009

We’ve taken on Nicholas Verge’s suggestion and sent another FOI request to OS about the international expert.

So once more unto the breach:

Dear Sir or Madam,

Thank you for your previous reply re the international expert who reviewed OS’s internal study on International Funding Models.

I note that the person does not wish to be named. I would like to request further details about the person, none of which I believe will require their identity to be revealed:

1) is the “internationally recognised expert (a) British or (b) a foreign national?

2) Is he or she a (a) currently employed in a UK academic institution, or (b) employed in the UK, or (c) employed in a UK government department, or (d) a UK government agency or trading fund.

3) If so, which institution, business/fund or arm of Government?

4) Has the person retired from any of those listed in (2)?

5) Has the person ever been employed by the Ordnance Survey and if so on what basis (eg full-time, continuous part-time, etc)?

5) Has the person ever been employed by a foreign National Mapping Association, and if so on what basis (eg full-time, continuous part-time, etc)?

Editing error: there are two questions 5. Let’s hope this doesn’t somehow invalidate the whole thing…

(I’m considering setting up a separate category for the IM/WOM. Any thoughts?)

Update: and just for completeness, we have asked Ed Parsons – formerly of OS, now of Google – whether he is the IM/WOM.

He responds, emphatically, that he is not. One less person…

Well, that was quick: OS responds to FOI re international expert; mystery deepens (if that’s possible)

Friday, July 31st, 2009

Following yesterday’s – yesterday’s! – questions to the Ordnance Survey about the identity of the international expert, we’ve had a response.

Here it is (emphasis added):

Thank you for your email dated 30 July 2009 requesting: the following information regarding the internationally recognised expert in Geographical Information and National Mapping, which I believe does not identify them personally.

We are pleased to provide you with the following information with regard to your request.

1) Does the “internationally recognised expert” work in a full-time or continuous part-time capacity for Ordnance Survey? – No.

2) If the answer to (1) is no, was the person formally commissioned on a contract basis by OS to review its study? – No.

3) If the answer to (1) and (2) is no, on what remunerative basis did the person review the study? – None.

So the international expert isn’t employed on any basis by OS, and reviewed the study for free.

Now I’m really fascinated. Who is this person? Why would they review this study for nothing? (Remember, we’ve ruled out Steven Feldman, Max Craglia and Robin McLaren.

Remember the definition: an “internationally recognised expert in Geographical Information and National Mapping”. More candidates, please. Or suggestions on how we can narrow their identity down further – while noting that they don’t want to be identified.

OS publishes almost unredacted version of international study

Sunday, July 26th, 2009

The OS has at last, in response to my FOI request (thank you whatdotheyknow.com) published an almost unredacted version of its international study, fully known as ” International Comparison of Geographical Information Trading Models – Study report”.

You can download it from the OS’s page about it, which contains the interesting addition (or is it that we only just noticed it?) that this is a study “which was commissioned by the Parliamentary Under Secretary of State for Communities and Local Government, Iain Wright MP, and formed one input to the Trading Funds Assessment undertaken by the Shareholder Executive and HM Treasury.”

Mm.

Anyhow, your opinions on what the study tells us – and especially whether it actually does manage to confirm any of the things that it was trying to confirm – are extremely welcome. Comments as ever are open.

That international consultant and man of mystery is… (updated)

Thursday, July 23rd, 2009

…is still a mystery. Ordnance Survey today responded at 18.18 BST (by my computer’s timestamp) to my request for the name of the international consultant who looked over the OS’s study justifying its own findings.

And the outcome:

I can confirm that Ordnance Survey does hold this information

I suppose it might have been done by anonymous peer review, but it’s unlikely..

however I regret to inform you that your request falls within the ‘Personal information’ exemption under section 40 (2) (a) of the Freedom of Information Act 2000 (FOIA). We believe this exemption applies because having contacted the person concerned Ordnance Survey has been requested to withhold their name in this instance, therefore we will not be releasing this information to you.

This person is turning in to some kind of remarkable beast. Let’s just remind ourselves of what we wanted to know: who is it that OS described as an “internationally recognised expert in geographical information and national mapping” who OS said had reviewed its study into how it should organise and charge for its service, and who “agreed with the analysis and conclusions”.

And now this person turns out to be such a shrinking violet that they don’t want to have their names in the papers and on the web?

This “internationally recognised expert” doesn’t want to be named? Doesn’t want to be named on a strategic study into one of the most important mapping agencies in the world?

This is, quite frankly, ridiculous.

We will now begin making representations to the minister ostensibly in charge of OS, Iain Wright.

And we haven’t even looked at the latest version of the study itself that arrived by email. Wonder what non-gems are buried in that.

Meanwhile, the search for Spartacus goes on…

Update: First: OS points out that since a recent reshuffle, Iain Wright is no longer the minister responsible; it’s now Ian Austin.

Second: following comments, I have put the question directly to Robin McLaren of Know Edge (mentioned in the comments below as a possible International Consultant and Man of Mystery). His reply, besides that in the comments below where he says Know Edge was not retained, is:

No, I have not been involved in the creation or review of the Ordnance Survey study. Like you, I am mystified as to who this ‘International Consultant’ may be.

Interesting: ‘severable improvements’ and derived data and Ordnance Survey

Wednesday, July 22nd, 2009

An interesting blogpost by cloudsourced about “Why OpenSpaces and GeoVation Vexes Me So“, talking about the key question – generally – about trying to build stuff using the Ordnance Survey’s OpenSpace API: why does OS want to demand that what you build with it becomes OS’s property?

Derived Data. The single most stifling element of the Ordnance Survey’s licensing regime is the practice of claiming copyright over any data derived from an OS base map. This has massive implications to anyone in a public sector body, or anyone who wishes to use their data. Any geographical data created by a public sector body (who almost exclusively use OS data) is not their’s to do what they like with, it is claimed by the Ordnance Survey under their copyright. Local authorities cannot share the location of its rubbish bins with the public, does this sound absurd to anybody else? Derived Data stops dead any effective sharing of public geographic data, and any innovation that would come from it.

This is a topic we’ve gone over and over. Interestingly, cloudsourced links to the terms & conditions of the OpenSpace API, which goes like this in section 5.4 on Derived Data:

5.4.1 You may create Derived Data, and You may permit End User’s to create Derived Data, in connection with Your Web Application. In the event that You or any End User creates Derived Data, such Derived Data shall be owned by Us

Flipping heck! Same old rubbish. But hang on, what’s this?

save that if any Derived Data is created which is a severable improvement (as defined by Commission Regulation (EC) No 772/2004, known as the Technology Transfer Block Exemption) of the Ordnance Survey Data then such Derived Data shall be owned by the person or entity creating the same.

Now the “severable improvement” bit caught my attention suddenly. Never noticed that before. But what is a “severable improvement”? It’s explained a little by crschmidt, who links to a discussion on the OpenStreetMap mailing list, where Richard Fairhurst describes it as

In practice, then, I read that to mean that if you use the Sustrans webmapping to find out where the routes go, this information is solely copyright Sustrans (who might be more willing to give permission) and not OS, even though it’s delivered through the medium of a derived work. So, if you had the wiggly lines already (whether mapped by GPS or NPE), you could tag them as NCN routes if Sustrans were ok with that.

My reading: if you’ve got something that didn’t come from OS, and which you can stick onto one of its maps wholesale, and remove wholesale, then it’s not – and cannot be – claimed as derived data by OS. But that doesn’t help if you don’t have it in some separate form; if you use OS as a base, then it’s derived data.

As Ed Parsons (formerly OS, now Google) points out in the comments

There is a relatively simple way to define derived information, and that is to ask what is not… If i create a new feature that is not represented as a feature on the original map/dataset it cannot be derived from it. If the OS would just agree with this it world more things forward.. but then we have been asking for this for years… At some point you have to move beyond cock-up territory to conspiracy…

We like Ed’s definition. It makes sense. I wonder quite what the effect on OS would be of adopting that definition. How sizeable a reduction in licence fees?

Sounds like fun: Society of Cartographers summer school

Monday, July 13th, 2009

(A public service announcement for our cartographic readers.)

Booking is now open for the Society of Cartographers Summer School at the University of Southampton from 7-9th Sept 2009.

If you want to:

  • hear the Director General of the Ordnance Survey talking about the future of the National Mapping Agency
  • hear about the new UKMap large scale map database
  • take part in a workshop on Google mashing, OS OpenSpace or OpenLayers
  • keep up to date on crowdsourcing
  • see how people are using OpenStreetMap data in the real world
  • hear the latest ideas on emergency mapping, transport mapping, 3d mapping and wayfinding
  • network with your peers and leaders in developments in cartography

then check out the programme at:
http://www.soc.org.uk/southampton09/program.htm
and register at: http://www.soc.org.uk/southampton09/index.htm

Naughty, very naughty: Ernest Marples frees the postcodes

Saturday, July 11th, 2009

An interesting new site – ernestmarples.com/ – is trying to make postcodes free.

The people behind it (the whois details tell you that it’s registered to a location in SW1A 1AA, which happens to be Buckingham Palace) are Harry Metcalfe and Richard Pope.

They insist, when asked the question of “where does the data come from?” that

We’re not saying. But, just to be clear: we don’t hold a copy of the postcode database ourselves, neither in complete form nor as part of a cache.

But their aim is clear enough:

Post codes are really useful, but the powers that be keep them closed unless you have loads of money to pay for them. Which makes it hard to build useful websites (and that makes Ernest sad).

So we are setting them free and using them to run PlanningAlerts.com and Jobcentre Pro Plus. We’re doing the same as everyone’s being doing for years, but just being open about it.

Hopefully the Government and Royal Mail will realise the value of this service and license us to offer it officially and for free. If not, and this website gets shut down, we’ll close the websites we’ve made that make use of this site’s lookup service. Permanently.

There’s a long list of people who have supported it. We’ll add our voice. The Free Our Data campaign thinks it’s a good idea to make postcodes freely available.

OS expert isn’t Max Craglia either… so who is it?

Friday, July 10th, 2009

You’ll recall the famous scene in the film Spartacus (directed of course by the same man who went on to direct 2001: A Space Odyssey) in which the Roman troops have captured the rebel slaves, and are trying to find out which of them is Spartacus, their leader.

At which one man stands up and says “I’m Spartacus!” And another, and another…

Well, the search for the identity of Ordnance Survey’s “internationally recognised expert” who looked over its calculations for its international comparison of mapping agency funding models is like that. Only in reverse. “I’m not Spartacus!” seems to be what people are saying.

In response to a suggestion in the comments that the person in question might be Dr Max Craglia, of the Joint Research Centre of the European Commission, a specialist in geographic information policies. So we sent off a quick email to him, asking if he was the one. (Don’t know who he is? main profile, another profile.)

“I regret I am not the expert you are looking for,” he responded, sounding more like Obi Wan Kenobi in Star Wars than Spartacus.

We’ve noted this in a roundup of what also happened at the Activate 09 summit, organised by the Guardian and part-sponsored by Ordnance Survey.

Among the other issues there were whether OS’s maps are fit for 21st-century digital economy purpose (Tom Watson MP, formerly of the Cabinet Office, thinks not) and also whether National Rail – the company owned by the train operating companies, rather than the nationalised success to Network Rail – should make its train running times available for free. Since it’s private-sector data, it doesn’t fall under the FOD campaign’s “government-owned or -generated non-personal data” umbrella.

Then again, the reaction on Twitter also suggests that with so many government billions being poured into the private rail sector, it would make sense to demand the data for free as a quid pro quo. It’s an argument that does have merits.

So in the meantime does anyone have any more (realistic) suggestions for who OS’s Spartacus is?

Michael Cross: setting data free is an easy promise when in opposition – so would a Tory government do it?

Friday, June 26th, 2009

Michael Cross, co-founder of this campaign, has an article at the Guardian’s Comment Is Free site on the Conservative pledges on data made on Thursday by David Cameron. Of note:

The three-year-old Free Our Data campaign – founded by myself and the Guardian’s technology editor Charles Arthur – will welcome Cameron’s re-stated promise to publish every item of government spending over £25,000 and raw data to allow communities to build their own crime maps and councils’ performance data in a standard format.

We will cheer most loudly at the plan to create a new right to data and proactively to identify the 20 most useful data sets on public services and make them available for web mash-ups.

But, he points out, there are warnings to be heeded.

To judge by Cameron’s speech, which makes no mention of the government’s single largest data business [Ordnance Survey], the Conservatives share this aversion to reform. The suspicion must be that the Tory solution is to try and sell off the mapping agency lock stock and barrel. Yet locational information is an essential component of nearly every public data set. To commercialise its supply would be to move in the very opposite direction of setting our data free.

It certainly is important for the Conservatives to set out clearly what their intention is with regard to OS before the election. A manifesto commitment not to sell it off would be a good idea.

Read the whole article for the wider points. Steven Feldman likes it – and adds

My one question is if the treasury are unable or unwilling to go down the centrally funded route what would you prefer – privatisation or trying to get the best out of the current model. I know which one I would choose.

David Cameron gives speech suggesting “setting data free”

Thursday, June 25th, 2009

David Cameron has given a keynote speech which continues to edge the Conservative party towards something that might look like the glimmer of the beginnings of the outline of the rough shape of a manifesto.

Part of it was to do with what he called “Setting data free”. See what you make of it.

In Britain today, there are over 100,000 public bodies producing a huge amount of information.

This ranges from school league tables to train timetables; from health outcomes to public sector job vacancies. Most of this information is kept locked up by the state. And what is published is mostly released in formats that mean the information can’t be searched or used with other applications, like online maps. his stands in the way of accountability.

(snip..)

… what about patient outcomes in the NHS? Some of the most important information you’ll ever need to know, how long your Dad will survive if he gets cancer, your chances of a good life if you have a stroke, all this is out of your hands.

Now, again, imagine if this information was in your hands. You’d be able to compare your local hospital with others, and do something about it if it wasn’t good enough. Choose another hospital. Voice your complaint to a patient group. Make change happen.

All this data which would help people in this country hold the powerful to account – it’s all locked away in some vault. And it’s only getting worse.


We’re going to set this data free. In the first year of the next Conservative Government, we will find the most useful information in twenty different areas ranging from information about the NHS to information about schools and road traffic and publish it so people can use it.

This information will be published proactively and regularly – and in a standardised format so that it can be ‘mashed up’ and interacted with.

What’s more, because there is no complete list that can tell us exactly what data the government collects, we will create a new ‘right to data’ so that further datasets can be requested by the public.

By harnessing the wisdom of the crowd, we can find out what information individuals think will be important in holding the state to account.

And to avoid bureaucrats blocking these requests, we will introduce a rule that any request will be successful unless it can be proved that it would lead to overwhelming costs or demonstrable personal privacy or national security concerns.

If we are serious about helping people exert more power over the state, we need to give them the information to do it. And as part of that process, we will review the role of the Information Commissioner to make sure that it is designed to maximise political accountability in our country.

The suggestion that councils will be obliged to publish data in a common format is one that Cameron has made before; and it’s something that Adrian Short’s mashthestate has been pushing (and successfully) without any political party.

The idea though of the “right to data” and the assumption that data should be available is interesting. Allied to Sir Tim Berners-Lee suggesting ways to get the data out there, it looks like all the political parties think that making data free – in the sense of untrammelled, if not yet in the sense of not-charged – is an idea whose time has come.

Using Gapminder to compare the countries studied by OS: how did they choose?

Wednesday, June 24th, 2009



See this screenshot on Flickr

Interesting to look at the countries which Ordnance Survey studied for its, er, study. Note that the study – which one would have hoped might be at least exhaustive, since it’s a one-off chance to really show things – excluded South Africa because it doesn’t have a comparable GDP per capita to the UK. Apparently comparable GDP was a specific measure used to decide who to examine.

Well, Gapminder offers the chance to analyse countries’ GDP per capita.

OK. So OS looked at Australia, Canada, France, Netherlands, Norway, New Zealand, Sweden, and the US – which, as Gapminder shows, do indeed have comparable GDP per capita. (This link may give you the graph – though you might have to click on the “map” link, and then go back to the “graph” link and play the runthrough.

But wait, what’s this? If you look at the graph, there are all sorts of other countries in there with comparable GDP/capita.

Even between New Zealand (the lowest) and Norway (the highest – didn’t expect that, eh?) there are loads of countries.

To wit: Bahrain, United Arab Emirates, Cyprus; Japan, Hong Kong. Monaco, Isle of Man, New Caledonia; Ireland, Denmark, Finland, Greenland, Sweden; Germany, Austria, Spain, Switzerland.

Anyone want to suggest any reasons for ignoring those? And what the business models are for their mapping agencies?

And it doesn’t make much sense if you compare GDP/capita against land area either.

Here’s the picture. Go analyse yourself – please. I’m too puzzled.

Ordnance Survey provides redacted version of its study into its financial organisation

Wednesday, June 24th, 2009

Ordnance Survey responded to our FOI request for publication of its study into itself and the best financial organisation for itself.

And in these times when MPs’ expenses are redacted, of course OS isn’t going to let us see everything.

That’s why the document, which I’ve uploaded as a PDF (though it’s originally a TIFF – apparently a scan of the paper document once the black marker pen had been wielded), is full of lacunae.

After some struggles (apologies) it’s now available as a PDF (3.7MB).

Have a look for yourself and see what strikes you as interesting.

This isn’t the final version of what we’ll get, however. In its response to me, OS says

Please see attached a redacted version of the International Comparison of Geographical Information Trading Models – Study report (ref: 71171). The report was commissioned by the then Parliamentary Under Secretary of State for Communities and Local Government, Iain Wright MP, formed part of an input to the Trading Funds Assessment undertaken by the Shareholder Executive on behalf of HM Treasury and the Department for Business Enterprise and Regulatory Reform (now Business, Innovation and Skills).

As you are aware this is an interim report while we wait for approval from the countries/agencies, who took part in the study, to release information related to them. The extension date for the next version of the report is 23 July 2009.

We’ll compare the two, of course. But it’s likely there will still be stuff cut out because it’s too sensitive for us poor souls.

OS first response to FOI questions: who did it talk to, and who helped? (Updated)

Tuesday, June 23rd, 2009

Ordnance Survey responded today to our FOI request. The redacted financial study is expected later today, but first here are the organisations that it talked to in deciding whose model to adopt. Or not adopt.

Here’s the text of the FOI response.

The report about which these question were raised, was commissioned by the then Parliamentary Under Secretary of State for Communities and Local Government, Iain Wright MP, and formed part of an input to the Trading Funds Assessment undertaken by the Shareholder Executive and HM Treasury.

1. Who or what was the “outside help”?

With regard to the International Comparison of Geographical Information Trading Models Study, outside help was provided by senior officials of those Institutions contacted.

In the case of the United States of America, as senior officials of the United States Geological Survey (USGS) were unavailable, Mr. David Cowen, Distinguished Professor Emeritus at the University of South Carolina, kindly provided us with an in-depth overview of the state of public sector GI data in the United States, including USGS. Mr Cowan is a former chair of the Mapping Science Committee of the United States National Research Council and is chair of the National Research Council’s Committee for the study of Land Parcel Databases.

The document was also reviewed by an internationally recognised expert in Geographical Information and National Mapping who agreed with the analysis and conclusions.

2. Which “equivalent organisations” were examined for the study?

There are no wholly equivalent organisations to Ordnance Survey, given its range of scales of mapping and other activities. Hence a representative sample of eight National Mapping Agencies or their closest equivalents were examined during the study. The sample included organisations with a wide range of data pricing policies: free, partial cost recovery (recovery of data dissemination costs), total cost recovery (recovery of data collection plus dissemination costs) and market price (cost recovery plus trading margin).

The overseas examples studied were:

Australia PSMA (Public Sector Mapping Agencies)
Canada Natural Resources Canada
France: IGN (Institut Geographique National)
Netherlands Kadaster
Norway Statens Kartverk
New Zealand LINZ (Land Information New Zealand)
Sweden Landmäteriet
United States USGS (United States Geological Survey) – via Mr David Cowan.

3. Which agencies did OS examine for the study?

IGN, Kadaster, Statens Kartverk and Landmäteriet have Agency status within their respective Governments.

4. Which agencies did the “outside help” examine for the study?

As indicated above, the outside help for the International Comparisons Study was provided by senior officials from the National Mapping Agencies examined in the Study, together with Mr Cowan and the internationally recognised expert.

And finally…

Please note that your enquiry has been processed to Freedom of Information guidelines. To the extent that the public interest (section 17) applies, we have determined that in all the circumstances of the case, the public interest has been met with the full provision of all information in this instance.

Update: Steven Feldman, who has been consulting for OS on how to get wider adoption of its OpenSpace API on its GeoVation initiative. Quoth Feldman (in the comments below): “We expect to go public on what we have planned for GeoVation in the next couple of weeks, watch my blog or mail me if you want to get a mail from me when we kick off.” (We apologise for the earlier error.)

, says he is not the “internationally recognised expert” mentioned by OS. We were going to ask but he beat us to it. The search goes on…

Reorganising this blog: different category suggestions

Friday, June 19th, 2009

Speaking to someone this morning about the Free Our Data campaign, I found that I was constantly saying “Oh, we wrote about X on the blog…” (where X could be the Postcode Address File, or the proposal to charge for changes in the Land Registry, or whatever) – but then realised it’s very hard to find anything by that label here. Search is nice, but it’s not enough.

Therefore I’m proposing to update the “categories” here into a number of new ones, principally with the names of organisations – eg Ordnance Survey, Post Office, Hydrographic Office, Land Registry – that would be affected or included in the post.

Your suggestions welcome for how, for example, we should deal with the “Cambridge report” or the Trading Funds review. And for any other organisational change to the categorisation on this blog for posts that would make your use of it easier.