Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens


Archive for 2006

In the Guardian: people are doing it for themselves

Thursday, November 2nd, 2006

The availability of out-of-copyright Ordnance Survey maps has given some people a smart idea: collect postcodes from visitors to the site and link them to the map points. It’s cheaper than licensing it from the OS and Post Office, after all.

Try it yourself at npemap.org.uk. There’s also, for those equipped with a GPS, the Freethepostcode site.

And a scheme to be launched next week will try to create an independent map of Britain’s roads, incrementally, from GPS systems in vehicles. But it would go beyond simple location data to include traffic flow data too.

Read about it at From postcodes to roads, we can collect it ourselves in today’s Guardian.

Ordnance Survey in the dock again with OPSI

Thursday, October 26th, 2006

This week in the Guardian we’re following the travails of the Ordnance Survey again. This time there has been a formal complaint about the terms it imposes for data about boundaries created in the Census:

The complaint concerns data delineating “output areas” from the 2001 census, the most recent national headcount. These areas are the smallest units in which data from the census is released; to protect anonymity, each covers about 125 households. By working at this scale, businesses can extract a huge amount of socioeconomic data to help them to decide where to open new outlets and even what products they should stock.

As with many other kinds of data, the most convenient way to display output areas is on a map. This is where the problem lies. The boundaries delineating output areas rely on data provided by Ordnance Survey, which as a trading fund generating returns from sales, treats intellectual property as a commercial asset. Anyone may view the boundaries online, but businesses wanting to include them in a product must negotiate licensing terms.

And you can probably guess where licensing of data ends up: with complicated terms and conditions that seem to involve chasing the Crown Copyright virus to the ends of the earth. Thus the Target Marketing Consultancy has made a formal complaint to the Office of Public Sector Information (OPSI) under the Information Fair Trader Scheme, to which OS subscribes.

The OS previously fell foul of an OPSI decision over land gazetteer data: OPSI found against it. OS was meant to release data – but we’ve not heard anything about the OS appeal against the decision (which it said it would consider). Anyone else know more?

Legal victory: statute database will be available to all for free

Thursday, October 19th, 2006

This week’s Guardian Technology reports that At last, the price is right for access to our laws – explaining that the legal statute database, which has been developed over years (and at some cost) will be available online for free.

We previously wrote about the pilot on August 17. Back then, there was a warning:

“The Statute Law Database and the material on the SLD website are subject to Crown copyright protection. The Crown copyright waiver that applies to published legislation generally does not apply to SLD because it is a value-added product.

But now, we learn via an email that was sent by Clare Allison, the enquiry system project manager at the Statute Publication Office, to testers of the Statute Law Database, that

the website as it stands will be launched free to the public once piloting has been completed.

However, there is a kicker – the government still wants (in the parlance of financiers) to sweat the assets by making them pay any way they can. The email continues:

A commercial strategy will still be developed next year, but will be looking at options that concern the commercial reuse of the data and the development of functionality that will serve the needs of the specialist user.”

Why not let commerce work out what it wants to do with the information, and just provide it, rather than trying to be both public and private sector?

Where we are today: chasing half a dozen ministers who won’t take responsibility

Thursday, October 12th, 2006

In today’s Guardian, Michael Cross looks back at more than six months of the Free Our Data campaign, and rounds up what we’re truly found out in this time.

Briefly, it’s that we have a £1 billion business which supports 25% of the economy – that is, public sector information – and yet nobody in government is prepared to take responsibility for it. By our count, there are at least six ministers (including Margaret Hodge, Ed Miliband, ohn Healey, David Miliband, Angela Smith and Pat McFadden) who are in some way or other in charge of public sector information. And none of them seems to want to be in charge of it, or take responsibility. There are two ministers in charge of the Ordnance Survey.

Read all about it at The fight has only just begun. And give us your advice on what we ought to do next. We wondered whether fixing up a few interviews with ministers might not help. And what would be the best questions to ask them?

Why aren’t public servants’ details public?

Thursday, October 5th, 2006

What are the penalties for releasing information gathered at public expense which is sold to the public? Why, huge fines and imprisonment. That’s the reality behind the Civil Service Year Book web page.

There, you can buy a copy of the book with names and phone numbers of thousands of civil servants for £70 – and get online access to the web page and its search function.

But if we were to make the username and password available to the public, we could be prosecuted for copyright infringement, with both civil and criminal liability. (If The Guardian did it, its editor could be sent to jail for up to six months.)

Does that make sense? If the information is gathered anyway, couldn’t outside organisations create better versions of it – rather as satnav companies make use of the data gathered by organisations like Ordnance Survey? Why create a monopoly in the data (by licensing it to The Stationery Office, a private venture between Apax Ventures and the TSO staff)? Isn’t it our data, since our taxes pay the people included and the collection of that data?

Other countries do make the information available – for instance in Seattle, where a listing of government staff is online.

Read more at Why Sir Humphrey won’t give us his phone number and give us your thoughts in the comments below, or by emailing us at tech@guardian.co.uk

Is France going to move to free geographic data?

Thursday, September 28th, 2006

In today’s Guardian Technology, the Free Our Data campaign looks at the example of France’s IGN – where 70% of its budget is paid by the government. That’s not producing the results that might be hoped for, says a damning report – and it recommends moving towards a model where the data is available for free online.

Far from encouraging the use of geographical data, the report says, the institute has discouraged the RGE’s take-up by setting high prices, despite a 70% government subsidy. The mechanism for setting charges is complex and secretive, relying on the “good sense” of administrators. Their incentive, is to get as much income as possible in the short term, which encourages squeezing more money from captive customers. Altogether, the inspectors find “a lack of rigour” in the institute’s commercial policies.

“This situation is responsible for the low level of sales and the feeble development of the geographical information sector in France, compared with other European countries,” they comment.

One problem is that government allows the institute to wear two hats, that of publisher and author. The report says that government has abandoned matters of geographical information strategy to the institute “allowing it to set policy according to its own vision and interests”.

Read France maps out the path to liberate its data for the rest of the story.

A million streets, four million holes in them: why not coordinate it?

Thursday, September 21st, 2006

In today’s Guardian Technology, Roadworks database is caught in a jam looks at the problems in putting together a database of where pipes and other near-ground objects are- which would be useful to utilities planning digs.

Is the roadblock secrecy? No. In the end it comes down to the problem of developing standards, and of organisations wrangling over intellectual property that they believe resides in knowing the location of those pipes. Put like that, does it make sense?

Even so, the problem of the NSAI (National Spatial Address Infrastructure) are key in causing the difficulties here.

Will revealing the exact location of mobile phone masts generate business?

Thursday, September 21st, 2006

The Information Commissioner is forcing Ofcom to release data about where radio masts used for mobile phones are located. If it happens, it could be an interesting test of whether free data does indeed generate businesses (even though this comes from businesses, not government.) Read more at Guardian Technology’s How can I find out where the nearest mobile phone base station is?

Times article echoes Free Our Data campaign

Wednesday, September 20th, 2006

The Times has printed an article by Gervase Markham of the Mozilla Foundation (which develops the Firefox browser, Thunderbird mail reader and a Sunbird calendar program) that makes exactly the points of the Free Our Data campaign.

He writes:

[a] double-charging scheme has been in place for years for government-collected data, such as maps, weather and hydrographics (rivers, tides and floods). After all, if our taxes paid for the collection, one would have thought that meant that we could have access to it. Right?

Until 1999, the Ordnance Survey, the British Government’s mapping arm, was funded by the taxpayer to make detailed maps of the entire country. These days, they sell limited-use licences to this national asset on a “cost recovery” basis. So, having paid my taxes, when I buy an OS map to go walking in the Lakes, I have the privilege of paying again.

Maps underpin many useful services we take advantage of today. From travel directions to house prices, much of the information we care about has a geospatial element. The effect of restrictions on mapping data availability are easily demonstrated.

The OS has, naturally, risen to its defence in the comments. But it’s nice to see commenters pointing out the existence of this campaign, and wondering which MP one should prod to get action. (My own comment pointing to this blog and the existence of the Guardian campaign doesn’t seem to have got past the moderator – newspapers can be so territorial; it’s as if rivals never have any ideas).

Certainly, the question of which governmental levers one should pull is uppermost in our minds right now, with Parliament about to come back. Who do you think we should try to interview to get the campaign more visible within government?

Has the National Archives got the right model for digitising old data?

Thursday, September 14th, 2006

Today’s Guardian Technology looks at the National Archives, and in particular the model it uses for digitising old data from past censuses. The contracts are let by tender to commercial organisations, which want to sell them on genealogy sites (ancestry is a huge online business). In return, they get exclusivity for a limited time.

The upshot: a thriving private sector, and a public sector which gets the work done for it yet also fulfils the challenge of making data available to the public. But it still costs..

The story is at National Archives squares the data circle. Your comments welcome.

Update: there was an important correction to this article, which appears on the link. Briefly, it was wrong to suggest that freedom of information might be suspended in favour of commercial digitisation. This was an editing error, and the suggestion was not in Michael Cross’s original article. (Following the links inside the article shows that NA did not suggest any such thing. We were wrong, and now it’s been corrected.)

Galileo is a publicly-funded satellite navigation system – so why were its codes secret?

Thursday, August 31st, 2006

In today’s Guardian, Galileo satellite’s secure codes cracked by Wendy M. Grossman looks at the mystery of the Galileo satellite system, so far publicly-funded, which is meant to offer public location services.. yet keeps its codes secret. Until, that is, an American team cracked them. Why aren’t the codes public, and can it keep them secret?

Why don’t local authorities release more data?

Thursday, August 24th, 2006

One of the topics brought up in the question-and-answer session at the RSA/Free Our Data debate in July was that local authorities often don’t know what data they can make available. SA Mathieson explores the topic more widely and explains how it’s a struggle to get data out of councils.

Sample extract:

But FoI has not created consistency in what authorities will release, although it is starting to help. Last year, local authorities including London’s Westminster city council refused to release hygiene inspection reports for food outlets under FoI, while the adjacent borough of Camden was putting such information online.

In December, the Information Commissioner’s Office (ICO), which has statutory responsibility for policing FoI, ruled that Bridgend county borough council in Wales should release a report for the Heronston hotel in Bridgend. The council had refused to do so, arguing that releasing these reports to the public would undermine discussions between businesses and the council on improving hygiene, but it decided not to appeal and now releases reports on request.

The intertwining of the Freedom of Information Act and this campaign is an interesting one, and we hear that the question of to what extent information released under FoI could be reused – perhaps commercially – is being discussed seriously within government. It would stop the strange situation allued to at the RSA/FOD debate, in which councils will release some maps but have to insist that they must not be revealed to anyone else. Freed information, indeed.

You can also now get a PDF transcript of the RSA/FOD debate.

Access to data denied: why isn’t there a free database of UK laws?

Thursday, August 17th, 2006

In today’s Guardian, Heather Brooke asks why there isn’t an online database which would tell us what laws are in force.

On August 2, the government rolled out the second stage of a long-delayed project to make the consolidated law of parliament accessible to the people. So how does it look? The public – who paid for the whole project – can’t get a look in.

No free public access sites have been granted permission to view the current system and testers of the database – predominantly from commercial legal publishing firms – have been told not to share their login and password. Even so, some testers are not entirely happy with what they’ve found after logging on to the top secret database of our country’s laws.

Firstly, an astounding Crown copyright notice greets the reader: “The Statute Law Database and the material on the SLD website are subject to Crown copyright protection. The Crown copyright waiver that applies to published legislation generally does not apply to SLD because it is a value-added product.

No matter that the value was added by public officials at taxpayer expense…

We think you’ll be amazed at how your data is being kept from you.

Manitoba, home of the free (data)

Thursday, August 17th, 2006

It’s always interesting to hear about foreign examples of data freedom. And the state of Manitoba, in Canada, has taken a very important step, after deciding that the bureaucracy invvolved in charging different bits of government for mapping data generated by another part of the state goverment was a waste of money – taxpayers’ money.

So it has made the data available for free.

Read Canada proves itself to be genuine land of the free: Manitoba (about three times larger than the UK, but with only 1 million inhabitants) might not give a precise comparator to the UK (since its local-level geography will change much more slowly, with less housebuilding and changing). But the rationale – that it’s too expensive to charge for, and more effective to make free – is encouraging.

You can also read about the Manitoba Land Initiative, as the scheme is called. [URL corrected – thanks Chris Corbin.]

So what is Inspire, and why is the UK lobbying against it?

Thursday, July 27th, 2006

In this week’s Guardian Technology, Mike Cross gives an overview of Europe’s INSPIRE directive, and explains why the UK government in particular is lobbying hard against the zero-pricing rules in it.

…Dr Max Craglia of the European commission’s joint research centre in Ispra, Italy, [notes] there is a two-metre difference between Belgium and the Netherlands in the official height of low tide – essential data for flood prevention. The anomalies multiply when many national agencies and tiers of government are involved, as can be the case when protecting stretches of coastline from damage.

…Nearly everyone supports the idea. But making geographical data freely available would destroy the business model of agencies such as Ordnance Survey, which funds activities by making a “profit” on sales of maps and geographical data. The OS warns of the threat in its latest annual report, published on Tuesday.

…The UK is unusually committed to charging users for data rather than funding its dissemination from taxation. One expert places Britain at the extreme end of the spectrum, while its system of crown copyright is unique in Europe.

(The OS’s latest reports, by the way, show a surplus of £9.58m on revenues of £118,356m. The capital employed – which John Bourn of the NAO still disagrees with, since he thinks the OS should capitalise its National Geographic Database – is given as £64m (if I’m reading the right column – tangible plus intangible assets). That’s a ROCE of 14.9%. Even if you add in Bourn’s estimated £50m for the NGD value, you still get a ROCE of nearly 7.5%. One has to think that the OS isn’t struggling yet.)