Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens


Archive for October, 2009

Digital engagement, widening and public data getting analysed… in private

Friday, October 30th, 2009

Stephen Timms reports that there’s been good progress in Making Public Data Public.

As the Digital Engagement blog notes:

So far our request for developers to “get excited and make things” has so far exceeded our initial expectations. Not only is the number of people signing up to the developer forum higher (currently more than 1,300), but also the discussion board is very active with a healthy list of ideas for the site and, perhaps most excitingly, a few applications are beginning to see the light of day.

And also:

Working in partnership with Guardian Professional, we held 3 developer days hosted at The Guardian‘s Kings Place offices in central London on the 14th-16th September. As an organisation they were best placed to help us undertake this task, having built a community of talented developers and opened up their API. You can have a look here at the excellent postcode paper concept and the rather wonderful traffic data visualisations here, which were just two of the many ideas for applications that emerged over the course of the camp. Ideas about their priorities for further data releases (to add to the 1,100 datasets currently on the site) were shared and important foundations for further iterations of the HMG Data site were laid.

There’s a certain irony in the fact that the sessions at the Guardian were held under such secrecy that I didn’t find out about them until the week after. More posts on that later…

Tim Berners-Lee to help UK government build single data access point

Thursday, October 29th, 2009

Computer Weekly reports that Tim Berners-Lee has been asked by the government to develop a single point of access for public data – as Stephen Timms, who has taken over where Tom Watson left off in the Cabinet Office, reports progress in “making public data public” (a concept that, when you think about it, seems a bit strange – as in “shouldn’t that have been done from the outset?”).

According to Computer Weekly, Timms told an RSA/Intellect event that

information is the “essential raw material” of a new digital society. “Government must play its part by setting a framework for new approaches to using data and ‘mashing’ data from different sources to provide new services which enhance our lives. In particular, we want government information to be accessible and useful for the widest possible spectrum of people.”

Well, minister, if that’s truly what you want, then you’ll make it free of charge, and free of copyright restrictions. It’s as simple as that. Could we suggest something like Creative Commons? The US government seems to find it amenable. .

Timms said, “We are supporting Sir Tim in a major new project, aiming for a single online point of contact for government data, and to extend access to data from the wider public sector. We want this project for ‘Making Public Data Public’ to put UK businesses and other organisations at the forefront of the new semantic web, and to be a platform for developing new technologies and new services.”

Fine words. We’d like some actions to go with them. We’re hearing plenty of sticks being wielded over how people use the net – Lord Mandelson’s threats to file-sharers, for example – but the carrots for companies to build on something that really would benefit Britain, by using British data, seems to be stuck on a really slow train.

Part of the problem, of course, is that it’s almost impossible to put a figure on how opportunity cost is lost through the lack of access to this data – whereas the music industry can much more easily point to figures it’s produced (though you may argue about their provenance) to suggest precisely how much harm it’s suffering through untrammelled downloading.

Interesting to contrast, though, that when we asked the Royal Mail to specify precisely how much harm it was suffering through the use by ernestmarples.com of the postcode to lat/long conversion, it robustly declined to say.

Of course there is the Cambridge trading funds report, with its analysis of the opportunity cost of the trading funds regime. But this goes much wider – the Cambridge analysis didn’t look at the Royal Mail and postcodes, for example, which have become embedded into many systems’ location processing.

Computer Weekly again:

So far, 1,300 people have signed up to the developer forum and contributed to the discussion board on what the data could be used for. The Cabinet Office also held a developers’ camp where ideas were shared.

We’ll have more about the devcamp in a future post.

Deleting users, for security

Thursday, October 22nd, 2009

As this site has been the target of some hackers (who were being paid by a Canadian pharmaceutical seller), I’m having to delete user registration here.

Sorry about that. Being hacked was a byproduct of having user registration, so one or the other has to go…

On the plus side, you do now get spiffy icons that will indicate whether you’re really the same person as when you last commented. So, not such a loss.

My thanks to Stefan Pause for his help and advice on securing the site.

And on the plus side, I’ve taken his advice and updated the permalink structure, which means you now get meaningful (if long) URLs. Win-win, I think. Plus I’ve got some fascinating hacker scripts to deconstruct.

Kent County Council wants you to recycle its data

Tuesday, October 20th, 2009


Have a look at http://picandmix.org.uk/:

Pic and Mix aims to increase public access to Kent-related datasets including those generated by Kent County Council (KCC). For the purposes of the pilot, we have brought together a sample of the most useful information. Where possible, it’s been provided in a format that allows it to be ‘mashed’ and customised. Please help us shape this initiative by suggesting additional data and ways in which we can improve this site. And if you do anything clever with the data, we’d like you to share that with us too!

The About page has more:

Last year, Kent County Council won Innovate08! Our idea had three elements:

  • To make publicly available information – things like crime statistics, employment information, business information – more
    accessible.
  • We also wanted to provide tools that would enable people to ‘pic and mix’ data to create customised information.
  • And last but not least, we wanted to provide a platform where people could share this information and discuss ways in which it could be used.

Winning Innovate08 meant we were given funding for a pilot project to see how people in Kent would respond to a resource of this kind. Our pilot project was intially launched with 25 small Kent-based businesses. With this new site we  hope to get the wider community involved.

So, how could Pic and Mix benefit you? Well, there’s a lot of information out there in a lot of different places. Rather than spend ages tracking down the information you need, we want you to come to a single place – picandmix.org.uk. For example, you may be looking for a care home for an elderly relative. You might want to mix this information with GP locations and bus routes. By plotting this information on a map you will be able to see which care homes are close to a GP surgery, and the bus routes. Another example might be a security company deciding where to focus its marketing efforts. They may want to mix office premises with crime statistics and use the information to plan a campaign.

Fascinating. We await developments – and news of same

Costing ernestmarples (and free data) vs paid-for

Sunday, October 11th, 2009

Somewhat late, but better than..

In the Guardian on Thursday, we have the cost-benefit analysis – if somewhat cursory – of having Royal Mail charge for its PostZon database (as used by ernestmarples, though indirectly) and having it available for free. So, for example, did RM lose out through ernestmarples? Or did we taxpayers benefit?

In Who would really benefit if postcode data were free, we add it up.

Royal Mail claimed that Richard Pope and Harry Metcalfe, the duo behind the site, had caused it “loss”. As the PostZon database being accessed via ernestmarples.com – named after the man who introduced postcodes to the UK – costs about £4,000 a year to license, could it be right?

Some simple calculations show that in fact everyone else, including the government that owns Royal Mail, and perhaps even Royal Mail itself, would benefit from the data being free.

Pope and Metcalfe point out that ernestmarples.com, which queried other websites that provide PostZon data for its postcode-location conversions, fed a number of their other websites – including Job Centre Pro Plus (which used a postcode lookup to find jobs near you), Planning Alerts (which alerts you to new planning applications in your area) and The Straight Choice (used to file election leaflets by area).

Job Centre Pro Plus had 437,354 searches for jobs since March this year, according to Metcalfe. If only 0.001% of those led to someone finding employment and saved £100 in benefit payments, then ernestmarples.com has, overall, saved the government money.

And Pope points out that professional property developers used PlanningAlerts “since it allows them to look for opportunities/competition”.

If that led them to work worth more than £20,000, the 25% corporate tax rate means the government has received more in tax revenue than it has lost from Pope and Metcalfe’s non-licensing of PostZon. Pope also notes that “few councils were using the PlanningAlerts API [programming interface] since it was easier and cheaper than paying external consultants to hack they achingly bad internal systems.” He points to Lincoln City Council, where PlanningAlerts was used to generate the RSS feed and map for planning. Would it cost more than £4,000 for Lincoln to build a system to do the job PlanningAlerts enabled?

Furthermore, “I was told by someone at the Electoral Commission that they used the Straight Choice during the Euro elections to monitor parties,” Pope said. “The alternative would be paying for hundreds of field agents (which they can’t afford).”

Rufus Pollock, a Cambridge economist who co-wrote a study for the government on the economic benefit of making trading funds’ data free, calculates that making PostZon free would bring an economic benefit 50% greater than Royal Mail’s present revenues.

Subequently it’s been suggested to me that the cost of licensing is more like £1,200 rather than £4,000 – which makes the case for benefit from free data even greater.

Royal Mail threat likely to close ernestmarples.org

Monday, October 5th, 2009

The Royal Mail has sent cease-and-desist orders (via its lawyers) to Richard Pope and Harry Metcalfe, the web developers behind ernestmarples.org (which we’ve referred to before).

Basically, they’re saying that ernestmarples is accessing the RM’s postcodes-to-coordinates database without permission, and that their clients are suffering loss as a result, and so they should stop.

Read it all (including the letter) at http://ernestmarples.com/blog/?p=3.

Two immediate questions:
1) how can it be unauthorised to access a database that is publicly accessible through others? Though of course there may be lots of fine print in white on a white background in which you “agree” not to reuse the information for anything actually useful anywhere.

2) precisely how much, and where, is the loss that Royal Mail has suffered? Ernestmarples would never have bought a licence. The services that they scraped for it (which RM’s lawyers have demanded a list of) are free to the public.

Tom Watson is ever so slightly incandescent about this; if he were a light bulb you probably wouldn’t be able to buy him in the shops.

This is stupid: as Tom Watson points out, closing this service will also affect other services that are being offered. Not clever.