Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens

Archive for March, 2007

Baroness Ashton listens to Free Our Data case

Thursday, March 29th, 2007

Baroness AshtonAs we report in today’s Guardian, in Minister listens to Guardian’s campaign call, we had an off-the-record meeting with Baroness Ashton, who is in charge of public sector information at the Department of Constitutional Affairs – a role that she has had since the Office of Public Sector Information moved from the Cabinet Office to become part of the National Archives near the end of last year.

The meeting was off the record (so we can’t tell you what she said) because the government is still considering its response to the OFT report on PSI, which won’t come out until after the local government elections in May. (We have no idea what the connection of the local elections is to PSI, but let’s let that go.)

One can though get some idea of the concerns that are going on; Baroness Ashton is clearly listening to everyone who feels that the current system isn’t quite working, as well as those who do. She has promised us an attributable briefing once the response to the OFT is published.

One concern we did note though was about how one could be sure, with a free data model, that errors weren’t being introduced in re-use:

Other questions raised familiar objections to the free data idea. One is how to assure the integrity of government data when it is re-used by third parties. While there are technical solutions such as encryption hashes, the answer may be that the government has to learn to let go. After all, in an era of free data, users and commercial competitors will always be able to make their own checks against the original. And free geospatial data could improve the accuracy of sketchmaps produced by bodies such as tourist authorities, perhaps even saving lives.

We can offer an example here: the photo on this page is, apparently, Crown Copyright; and anyone can follow it back to the original, here, to see if we’ve messed around with it. (We haven’t, because we’re simply calling it directly from the government server.) Data re-use with Crown Copyright without cost? It sounds like a good idea to us.

In The Guardian: a year of Free Our Data campaigning: why is the Office for National Statistics free?

Thursday, March 22nd, 2007

And what have we got? You might ask. Plenty of interesting points that there wasn’t enough room to fit in print, in fact (we didn’t get into the matters of other countries which don’t charge for their mapping).

Among a list of “interesting things we’ve learnt in the past year” in A few victories, but the battle goes on in today’s Guardian is one that has been intriguing me ever since the NCeSS event last week.

It’s this: the ONS makes its data available for free. The Ordnance Survey doesn’t. But they’re both dealing with data that are constantly changing.

As the story puts it:

  • Despite being in effect half taxpayer-funded, OS’s position as a trading fund protects it from financial neglect by the Treasury, according to advocates of the model. They argue that OS has to collect data about a constantly changing landscape, and that making it fully taxpayer-funded would put it at the mercy of central funding, which could wane (as happened between the two world wars).
  • The Office for National Statistics, which collects data about constantly changing social elements such as internet use, labour, manufacturing output and so on, is not a trading fund, and makes the majority of its statistics – including demographic data from the 2001 Census – available for free via its website.

So here’s the question: why isn’t ONS a trading fund, if it’s such an effective model?

Your questions please for Baroness Ashton – and a question for you, the reader

Tuesday, March 20th, 2007

Baroness Ashton, of the Department of Constitutional Affairs, and about as close as the government gets to being a minister for public sector information, has agreed to give us an interview meet us, off the record, next week. She would like to “hear our concerns”.

So, what questions would you like us to ask her? Reasonable ones, please.

And (since I’ve linked to theyworkforyou): Tom Steinberg, one of the people behiind MySociety, asked a key question about the realpolitik of the campaign:

“What single piece of evidence would you put forward to the Treasury that would convince them that it is worth spending tens of millions of pounds to make this data free?”

In other words, where’s the proof that this move to free data would really bring economic benefit?

Your submissions please. Links to papers (even very geeky economic ones) for, and against it you like.

‘What happens at the next Lockerbie?’ – the risks of killing NIMSA

Tuesday, March 20th, 2007

One of the points that I made during the Open Knowledge Foundation meeting in London last week was that the Ordnance Survey knows when everything changes. It has a mission to map the UK, and pretty much anything the size of a garden shed will get noticed by its overflights. (Wouldn’t local authorities love to know about changes in their areas that conformed or didn’t to planning permission? How much do they have to pay if they do?)

I’ve been to OS – which apparently six of the eight past ministers in charge of OS haven’t – and seen the work they do loading the overflight data onto the MasterMap. It’s impressive. The OS target is to get 99.6% of changes in the database within six months.

However the end of the National Interest Mapping Services Agreement (NIMSA) last year means that the OS gets no subsidy to map areas that are out of the way. If it’s having to compete with a growing number of commercial services (apparently the latest one Vanessa Lawrence is concerned about is China’s mapping agency), how can it justify mapping remote areas at that speed?

Ed Parsons, former chief technology officer at Ordnance Survey, says that it won’t. “Areas in cities will get updated, but in Scotland your new garden shed might not be noticed for five years.” Nice for your garden shed – but what happens when a plane or a tanker or some other disaster happens in that remote area that has been neglected because of the death of NIMSA?

That is why the Free Our Data campaign says that Ordnance Survey is valuable – and that the government has a responsibility to citizens to make sure the UK is well mapped, within the public sector. Duncan Shiell of OS, who spoke at the NCeSS event, said that between the wars, councils did a lot of the mapping – but that when the OS was re-funded back to strength and took it over, it discovered that many of the maps didn’t join up across county boundaries... explains what happened between the wars in the comments (Any errors are mine, from misremembering.)

We think the point remains though. That’s why you need a well-funded – taxpayer-funded, not privatised – OS.

ESRC events and Open Knowledge day: podcasts/transcripts available

Tuesday, March 20th, 2007

The Open Knowledge Foundation event was very interesting: there’ll be a couple of blog posts arising from it (which I’ll put up here in a moment).

Meanwhile, Steve Coast of OpenStreetMap has put up the podcast of the panel at which Ed Parsons, he and I (Charles Arthur) spoke.

The National Centre for e-Social Sciences (NCeSS) has a page which might, or might not have the webcast – it’s not clear whether it will be archiving the content. It was a very interesting debate, made more interesting by the fact that many had thought – wrongly, we emphasise – that the campaign wants to make personal data available. We don’t. The aim is to get impersonal data made available without restriction for the cost of reproduction.


an interesting writeup from an archaeologist’s point of view.

Free Our Data: sessions this Thursday and Saturday, in Manchester and London

Wednesday, March 14th, 2007

The debate continues over free data: I’ll (Charles Arthur) be appearing this Thursday at Manchester University, with the National Centre for e-Social Science.

The Manchester event is in the evening, starting at 6.30pm: I’m on a panel with Peter Elias, Professor of Labour Economy, University of Warwick and Strategic Advisor (Data Resources) to the ESRC; Duncan MacNiven, the Registrar General for Scotland; Jil Matheson, director of Census, Demographic and Regional Statistics at the Office for National Statistics; and Neil Ackroyd: Director of Data Collection and Management, Ordnance Survey.

The topic:

Should social researchers have unlimited access to the data that Government collects about individual UK citizens? Would the benefits of better evidence on which to base social policy outweigh concerns about privacy? Experts from academia and government will offer their views and respond to questions from the audience.

With those attendees, it should be quite an event..

On Saturday, I’ll be on a panel at 11am as part of the Open Knowledge Foundation conference in London

with Ed Parsons, until recently the chief technology officer of Ordnance Survey, and Steve Coast, founder of OpenStreetMap.

Read more on that, including directions etc, at the OScon site.

Free our address data – or at least get them to stop charging each other: the petition

Tuesday, March 13th, 2007

As noted in this comment elsewhere on the blog, there’s now a No.10 petition to stop the address madness:

We the undersigned petition the Prime Minister to stop the expensive and damaging address ownership conflict currently existing between Royal Mail, Ordnance Survey and Local Government by definitively establishing that the intellectual property input of all three parties into derived address products such as PAF, AddressPoint, NLPG and NSG is equal in scope and value. Consequently, in the public interest, none of the three parties should charge either of the other parties for use of these products.

While we’re sceptical of the power of these petitions to change anything, they’re a good meeting point to indicate strengths of feeling.

So – sign it and then meet back here to work out what financial impact this would actually have – especially since local authorities are meant to be planning how much to charge for searches relating to homes (link to Yvette Cooper MP dithering on the matter earlier this month).

If they didn’t have to pay for that part, would searches be cheaper?

Geology is free (well, will be)

Friday, March 9th, 2007

From the Guardian of March 9, Geological knowledge to go online:

British scientists are leading an international effort to bring together all the known geological information about every country in the world. By making the data freely available and allowing researchers to track geological features across national boundaries, the project will make it easier to plan international projects, predict earthquakes and locate natural resources such as oil and gas.

Once the project, called OneGeology, is up and running the data will be searchable via the internet. “Geology has no respect for national frontiers,” said Ian Jackson, who is coordinating the project for the British Geological Survey (BGS). “The data exists, but accessibility is the key.”

We’ll be interested to see how it copes with saying precisely where these geological features are in the UK without reference to Ordnance Survey data; or perhaps it’s going to license it. In which case who’s going to fund it?

I’m reliably told too that the Free Our Data campaign was mentioned at the launch – and that while Ireland is going to make all of its data at all scales completely free, the same won’t be true for the UK, where you’ll have to pay for data below a certain scale.

That noise in the background is me, grinding my teeth.

In The Guardian: how and why South Africa set its data free

Thursday, March 8th, 2007

Following on from our earlier post about South Africa’s free mapping data, we got in touch with Derek Clarke, head of the CDSM (South Africa’s mapping agency) to ask him what effect the imposition of zero costs – a corollary of the 2000 Promotion of Access to Information Act. (That’s a Google cache link, in HTML.)

In South Africa’s freedom includes its data, we ask whether the move from a charging system, which used to bring in 5% of the budget, has brought any benefits – either in reducing waste, or expanding use, or both.

Derek Clarke, head of the agency, says yes to both: most of the clients for the CDSM’s mapping data before were government agencies, and in Clarke’s pithy words “government paying itself makes no sense but causes administrative waste.” And has use grown? Yes, by 500% (we make that sixfold), he says.

Is it a model for others? Clarke again:

“This model should be applicable to all developing countries where the government must play a developmental role. The same situation does not apply to developed countries with mature markets. However, governments of developed countries should evaluate the opportunity cost of geospatial data – it may be more beneficial to make data free.”

We’ve put the questions and answers from Mr Clarke online already, but the article aims to deal with the critiques we know will follow from any suggestion that the UK might have something to learn from South Africa…

Derek Clarke, head of South Africa’s mapping agency, responds to our questions

Thursday, March 8th, 2007

We earlier noted some questions we had about South Africa’s free mapping. So we emailed Derek Clarke, the head of the Chief Directorate, Surveys and Mapping. Here are our questions, and his answers.

  1. How is the CDSM’s funding guaranteed, and is it enough to keep up to date with changes in South Africa’s geography?
    Funding is 100% from the Parliamentary vote. It is not enough to satisfy all client needs – but must be seen in the context of affordability for the country.
  2. How much has the move to a ‘free data’ model cost, compared with the revenues you received before from selling digital data? Or have you found a way to compensate for that by increasing revenues from other sources?
    Previously the revenue generated was approx. R3.5 million [£243,000 at current exchange rates – CA]. It has had no impact on us as all the revenue had to be returned to the central revenue fund of government. Previously there was no incentive to sell data, except for the satisfaction of knowing that you have happy clients. It should be noted that most of our clients are other government departments and therefore the money paid is government money – government paying itself makes no sense but causes administrative waste.
  3. Is this [free data model] sustainable?
    If we did retain revenue generated it was insignificant compared to the operating budget – approx 5%
  4. Has the number of organisations, both public and private-sector, taking the CDSM’s digital data increased with this move?
    Yes, by about 500%. This figure could be inflated because some requests could be over-serviced (getting more than what the client really needs)
  5. Do you think that the economy as a whole has benefited from this move (that is, have the taxes generated by private-sector companies

using your data, or savings made through using it, compensated for any loss in revenue from charging for digital data)?
Yes. This was one of the motivating factors to make data free – to stimulate the private sector in providing services and to reduce their input costs. The spin-off being job creation.

  • Do you think this model is applicable in other countries, such as the UK?
    This model should be applicable to all developing countries where the government must play a developmental role. The same situation does not apply to developed countries with mature markets. However, governments of developed countries should evaluate the opportunity cost of geo-spatial data – it may be more beneficial to the country to make data free. Please note that the price of data and copyright are two different issues. It is possible to have free data but still have copyright on that data. Copyright on free data is not used to restrict the use of the data but rather to ensure that the user acknowledges the source and also to ensure that no other party claims a copyright on that data.
  • We’ve also followed up in this story in The Guardian Technology section.

    South Africa: mapping is free (and so is other government information)

    Sunday, March 4th, 2007

    The opening of the South African government’s Freedom of Information Act 2000 states its position bluntly:


    • the system of government in South Africa before 27 April 1994, amongst others, resulted in a secretive and unresponsive culture in public and private bodies which often led to an abuse of power and human rights violations;
    • section 8 of the Constitution provides for the horizontal application of the rights in the Bill of Rights to juristic persons to the extent required by the nature of the rights and the nature of those juristic persons;
    • section 32(1)(a) of the Constitution provides that everyone has the right of access to any information held by the State;
    • section 32(1)(h) of the Constitution provides for the horizontal application of the right of access to information held by another person to everyone when that information is required for the exercise or protection of any rights;

    ..and the upshot: maps of South Africa are free. Have a look at the maps pricing page: the only charge is for postage and packing. Which on the digital products is zero, though with the proviso that the supply of digital information not contained in the off-the-shelf products and/or required in any other format, an hourly rate will apply to prepare such information: Provided further that the Chief Director of Surveys and Mapping can provide the data in the required format.

    Questions we’d like to know the answers to:

    • how much does it cost to do this?
    • is the mapping agency’s budget assured, and if so, how?
    • how is the quality of the mapping data assured? (We can have a guess at this from the fact that there’s a network of GPS stations, whose locations are available for download, for free)
    • Has this led to any measurable increase in mapping services in the private sector?
    • Has it had any negative or beneficial effect on mapping suppliers – including the government mapping agency?

    We’ll be trying to contact the SAMA, but if anyone has any pointers to answers to these questions, we’d be grateful for the information.