Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens

Archive for January, 2009

February 10: come to the public debate on free data, government agencies and copyright

Friday, January 30th, 2009

On Tuesday 10 February at 5pm, the think tank Policy Exchange will be hosting a debate entitled Free Our Data? Government Agencies and Copyright, in Westminster.

Speakers will include the Technology Guardian editor, Charles Arthur, a co-founder of the Free Our Data campaign; Adam Afriyie, shadow minister for innovation; and Ed Parsons, geospatial technologist at Google and former chief technology officer at Ordnance Survey. We’re hopeful that there will be at least one other speaker, from the public sector.

If you would like to attend, email (You’ll discover the precise venue then!)

Wouldn’t it be wonderful to have this data for the UK?

Sunday, January 18th, 2009

The Guardian has pulled together a collection of datasets drawn from the US:

Simon Rogers gathered this information and shared the raw data via Google Spreadsheets for anyone to use. This means that people can grab the data in whatever format is most desirable including text, .csv, .xls, and .pdf.

Since access is open on each spreadsheet, it also means that developers can write client applications that interact directly with the data. Developers can access the same source data as either XML or JSON.

Fantastic stuff. Now, wouldn’t it be marvellous if we could find the same datasets for the UK and know that we could all share it for people to build on?

If you do know of any copyright- or charge-free (or both) sources for these for the UK, then please leave us a note in the comments.

In the Guardian: the mystery of the vanishing addresses

Friday, January 9th, 2009

Thursday’s Guardian Technology looks at the mysterious ways in which addresses within some postcodes are simply vanishing from the Postcode Address File (PAF) – that enormously useful index of places in the UK which can receive mail. A new Highland clearance? Well, sort of.

In January 2008, the picturesque west Highland village of Applecross contained 32 buildings with postal addresses. A year on, it has only 24. This is not the result of some new Highland clearance, but an absurd consequence of UK government bodies treating data collected in the course of their work as a commercial asset rather than a national resource to be shared.

It’s not that the houses are going away; they’re very much still there. And they’re still owned by people. But Royal Mail, in attempting to maximise the value of the PAF, is removing them because that makes PAF more valuable to direct marketing companies – even while it reduces its utility to local authorities, which initially gave Royal Mail the details of the addresses, because they need to know about habitable locations in order to do things like emergency planning and other local services (dustbin/recycling runs, anyone?).

Royal Mail says it has a policy of removing addresses from the database when houses are unoccupied. “If the postie can no longer reach the delivery point, or if a house is obviously completely unoccupied, the postie informs us and the address is removed from the PAF. If it later becomes occupied, it would be put back on.”

And then…

Turf wars between Royal Mail, local authorities and Ordnance Survey over the ownership of postal addresses have a long history, imperilling everything from emergency services to the national census. Local authorities are particularly bitter about the current state of affairs because they have the statutory job of creating addresses in the first place. As one council specialist put it: “Local authorities create addresses, Royal Mail adds the postcode – then this data is sold back to us by Royal Mail and Ordnance Survey.”

Any other examples of this that anyone has come across?

A quick roundup to start the new year

Thursday, January 1st, 2009

Hope you’ve all come through the new year without suffering too many leap-last-year problems. I thought it would be interesting to round up a few things that I’ve seen but not really had enough brainpower to turn into anything more than notes.

First, Public Data Sets stored on Amazon Web Services. (Via Richard Allan.) An interesting idea: got public datasets? Well, why not get them stored somewhere really cheap where people can access them but you only pay per download. It’s the ultimate outsourcing, and you also get to see how many people are downloading it without the capital costs of the servers.

Public Data Sets on AWS provides a centralized repository of public data sets that can be seamlessly integrated into AWS cloud-based applications. AWS is hosting the public data sets at no charge for the community, and like all AWS services, users pay only for the compute and storage they use for their own applications. An initial list of data sets is already available, and more will be added soon.

It’s already got the Human Genome and the US Census data. The idea of hosting UK public datasets on AWS was floated in the Cambridge Economics report released with the Budget back in March. Any takers?

Second, Municipalities open their GIS systems to citizens (thanks, Gerry Gavigan, who points out that “As well as innovation and the other usual unexpected benefits, it points to the existence, alas without quantification, of financial benefits.”) The article explains:

For instance, the online burning permit sales service of the Minnesota Department of Natural Resources (DNR) allows citizens to declare precisely where they would like to burn woody debris. High precision is essential in deciding whether a permit is obtainable, as well as when and under what conditions: if there is a high fire risk in the area and day for which a user asks for a permit, the software must refuse it. The Web site, however, makes it easy to enter the location with the greatest possible resolution: users first type an address into a form to get an approximate location on the map, then zoom at will and finally click on the exact spot for which they are applying for a permit.

And, more pertinently:

The success of initiatives like OpenStreetMap or the availability of Yahoo! and Google Maps APIs may make you think that people may create services like these and many more all by themselves, without getting any bureaucrat involved. However, in order to benefit the most from digital maps and other spatial data, citizens need such data to be officially inserted in, and completely integrated with, the maps and databases public administrations use to plan roads, zoning, and everything else.

Citizens may use Web sites like those mentioned here to request services as different as bus stops, trekking permits, or new post offices. Other uses may include signalling construction abuses, damages to public property, or illegal dumpsters. We may draw our preferred public bus routes on a map in our City Council official Web site.

Of course, to make all this work in practice, public administrations should also clarify the data ownership situation. Who owns data directly and freely provided from citizens? What license should apply to those data or any derived ones? This, however, is a separate issue, not really related to open source software.

And finally, some interesting questions being asked in Parliament by John Howell (of the Tories) about Ordnance Survey income from local authorities, and on use of OS vector data for commercial use (and, previously, about discussions between OS and Google over mapping licences; Ed Parsons, formerly OS and now Google, says the minister’s answer is wrong); while Mike Gapes, a Labour MP, about London mapping payments to OS and payments to OS for use of its data by various government authorities.

We’d be interested in any comments on what Gapes and Howell are trying to unearth here… and of course your comments on anything else. And happy new year!