Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens

Archive for June, 2007

Environment Agency yanks flood data from OnOneMap site

Saturday, June 30th, 2007

We’ve written in the past about OnOneMap], which took the interesting step of taking the mobile phone mast data from Ofcom and making it available on its property/rental search site.

But recently it did something much more interesting – and, given the weather, useful. At the start of June it added the Environment Agency’s flood risk data to its Google Maps implementation.

The Environment Agency’s flood data is, to be honest, not as useful as it could be. You can do a postcode search, but when you then look at it, it is difficult to work out quite what’s at risk.

An article in today’s Guardian – Agency’s flood maps fail to hold water – makes this point more elegantly. It appeared in the Money section of the paper.

As the article points out,

users will find the site lacks crucial details. For example, it fails to show the location of a home in relation to the area at risk of flooding.

OnOneMap managed to grab the Environment Agency data (for England and Wales; we’re checking on Scotland) and added it as a layer to its Google Maps.

The the Environment Agency got in touch: the data, it asserted, was its copyright, and it wasn’t happy about it being used in this way – even though OnOneMap is not (at present) a for-profit site. It asserted its ownership of database copyright in the data, which is hard to rebut, and threatened to take OnOneMap to court.

(As I write this, the news is on, saying that the estimated costs of the flooding this month are £1 billion.)

Without the resources to fight such a battle, OnOneMap removed the data – but not without making a note to that effect on its blog:

The Environment Agency claims they have copyright over the information, and despite the fact that tax-payers’ money has paid for it to be collected in the first place, apparently the tax-payer cannot benefit from innovations like our housing and flood map combination.

The comments on the post are quite illuminating, for example:

This is absolutely outrageous given our tax money has paid for this. Surely it is up to the government agencies to ensure this information is widespread, ESPECIALLY during this time period when we are being inundated with water!!


Ridiculous, especially right now, that people need to find out if their home is at risk of flooding… greedy buggers, the Environment Agency… I loved the feature on, shame you had to withdraw it

Interestingly, there is a site which does have very detailed data – which was gathered by the Norwich Union – is you have to pay for the data (unsurprising, since it cost Norwich Union something like £5m to gather it..). But wouldn’t it be better if the Environment Agency data was available to all of us, free, without having to go to its site?

In today’s Guardian: Galileo to be publicly-funded – but why compete with GPS?

Thursday, June 21st, 2007

Today’s Guardian Technology section wonders “Will Galileo ever achieve orbit?” That being the project to create a GPS-alike system, but funded from Europe rather than the US.

The costs involved are large –

a 2001 report commissioned by the EU estimated that developing and deploying Galileo would cost €3.4bn (£2.3bn). Philip Davies, senior account manager at Surrey Satellite Technology Ltd (, says he’s seen estimated running costs in the range of €8bn to €10bn over 20 years. These numbers will be part of what is scrutinised between now and September, the deadline the Transport Council has given the European Commission to come up with alternative proposals for funding and managing the system.

GPS-style systems have the benefit that they’re government-generated data which is made for free: that creates private-sector opportunities. Very large ones:

The business plan published by the Galileo Joint Undertaking at the outset of all this estimated that the market for satellite navigation applications would grow from €30bn in 2004 to €276bn by 2020. This estimate was conservative compared to some of those in the report the EU commissioned in 2001 from PricewaterhouseCooper, which projected a market of €276bn by 2015 for personal communications and location services.

Clearly there’s a strong multiplier at work in GPS – comparatively small government input produces big private-sector revenues (and hence taxes, which pays for GPS – in theory – and all sorts of other benefits, as well as reducing congestion and making people easier to locate if they’re in danger, and stopping planes colliding; putting a value on the “not happening” bad things is hard, but must be counted as part of the benefit of satellite location.)

Galileo remains something of a me-too, but it’ll be interesting to see whether the EU decides to focus on its public and private-sector benefits as a justification for the spending – or whether instead it just kills it. Somehow, it doesn’t look likely to kill it.

New Zealand makes statistics data free to encourage business – but where’s the logic?

Friday, June 15th, 2007

In an announcement that we’re struggling to find much coverage of, New Zealand’s statistics are to be made available for free starting from this August, down from prices of NZ$25,000 (£9,500) in some cases.

That’s quite a radical move, explained by the NZ minister for statistics Clayton Cosgrove thus:

“I am pleased to announce that information to help businesses identify market opportunities, assess their competitiveness, and implement informed investment planning will be made freely available. The roll-out of information will include a host of industry-specific information for the building, retail and tourism sectors, and for importers and exporters. The data will also be useful for local authorities and communities,” Mr Cosgrove said.

And also – chiming with the Free Our Data rationale –

“Previously the information could be ordered at a cost from Statistics New Zealand, but in future, trade figures, for example, which were charged out at around $400 per customised request, or Digital Boundaries files that cost up to $25,000, will be available free.”

“The Labour-led Government is committed to giving businesses every opportunity to grow and prosper by providing the tools to support well informed decision making. Making key information available at no charge will encourage more businesses to identify new markets, for example, and plan for the future.”

Politics aside, businesses are keen on it:

Phil O’Reilly, Chief Executive of Business New Zealand, said, “Business groups have consistently advocated that this valuable information be made freely available, as it is in Australia. I am pleased the Government has taken this step.”

Well, yes, anything that reduces a direct business cost is going to be welcomed. But of course this will have to be balanced with increases in the amounts spent on direct taxation: the NZ government is putting NZ$6m into Statistics NZ over the next four years to ease this process.

And of course in the UK, National Statistics makes all its raw non-personal data (correct me if I’m wrong) available for free, except where it’s forbidden by copyright involving mapping agencies, and only charges for custom-made datasets.

There’s some media coverage – New Zealand Herald interviews the head of Statistics NZ.

An interesting NZ government press release from August 2006, when Mr Cosgrove talked about how statistics can help small businesses. (But not if they can’t afford them, eh?) And the Salvation Army pointed out that the free data is good news for the charity sector:

‘Many of the statistics are currently prohibitively expensive for non-profit groups, so removing the charges will make available a wealth of information otherwise inaccessible.’

A full list of what’s being made available (the government press release plus accompanying PDF).

And here’s a key part, from the FAQs: the expected growth.

Have other countries done this?
Yes. Australia and Denmark have both seen big surges in use of data following similar initiatives to make statistics freely available. The Australian Bureau of Statistics reports data downloads have approximately tripled since they made similar information free in 2005.
What is the uptake of the data expected to be?
A similar upsurge in data uptake is expected in New Zealand. In 2003 Statistics New Zealand made Census information freely available on the internet. This has resulted in a significant increase in public usage from around 250 paying subscribers in 1993 to over 20,000 accesses in the last year alone. [Emphasis added – CA.] The INFOS system currently has 93 annual subscribers. Once the system is redeveloped for easy use on the web, based on international experience, usage could increase to between 1500 to 2,000 users per month, and businesses would become the predominant sector using the information.

What we can’t find is anything leading up to the announcement, nor any indication of the economic analysis that surely must have been done before embarking on this. Any pointers?

Minister confirms government response to OFT on PSI by end of June

Thursday, June 14th, 2007

Wondderful thing, theyworkforyou – it takes government (well, Parliamentary) data and repurposes it to create something more useful.

We’ve just noticed that among the written questions is one by Mark Todd, who wanted to know when the government will respond to the OFTs’ CUPI report.

The response of Ian McCartney, DTI minister:

The OFT’s report contains some challenging recommendations. Given the importance and potential impact of the recommendations, and the wider constituency of interested parties both within and outside Government, more time was required to properly frame the response. The Government expect to publish their response to the report before the end of June.

We’re intrigued by the idea that there are “challenging” recommendations in the OFT report. But at least this gives us a clear indication that things are moving.

In today’s Guardian: more examination of ‘The Power Of Information’

Thursday, June 14th, 2007

Today’s Guardian looks at the Ed Mayo/Tom Steinberg report The Power Of Information and asks what sort of Whitehall it would be that could open up, and what the effects will be.

At the moment, the government’s attitude to the web is a mixture of aloofness and outright hostility. This should change, the report argues, partly because some of the most popular user-driven communities – MoneySavingExpert, for example – are closely linked to government policy. Others directly contribute to the public good: in Los Angeles, when the government started putting the results of food safety inspections online, the incidence of food-borne illnesses fell compared with that in neighbouring jurisdictions.

The government should also be involved because online communities are big users of a repository of data generated by public bodies ranging from tide tables to school league tables. The internet greatly increases the value of this information. The humble postcode, originally developed for a single purpose, now underpins countless public, private and voluntary services.

We’ve been around postcodes before, of course, but what’s interesting is that it has become a datum whose use has expanded far beyond its original intention by virtue of being mashed up with something else – geographical location. (In fact we might call postcode analysis the first mashup.)

However, Government 2.0 is not yet official policy. The Cabinet Office will respond “in due course”, officials said, almost certainly to coincide with the government’s overdue response to the Office of Fair Trading’s report on the commercial use of public-sector information.

As we said – long overdue. As in three months overdue. But it makes sense to lump these together, even if the Mayo/Steinberg report took only one-eighth the time to write by my estimate.

The snag is that the response will need approval from arms of government whose income is likely to be hit by the proposals. If Ordnance Survey or the Meteorological Office had to give away information for which they charge today, they would look to their sponsor departments, Communities and Local Government and the Ministry of Defence, to fill the gap with tax revenues. With a tight three-year spending squeeze to be launched by the Comprehensive Spending Review in October, this would not be popular.

Hmm.. but on the plus side, the report recommends that

• By March next year, the government independently review the cost and benefits of supplying public information through trading funds. This would examine the five largest trading funds, the trade-off between revenue from sales of information, the wider economic benefits of giving the data away and the potential impact on the quality of data.

• Public bodies, including trading funds, only to charge the marginal cost of distribution for raw information – which online is usually zero. The only exceptions should be where independent analysis shows that this does not serve the interests of citizens.

• All trading funds consider introducing free licences for non-commercial re-use of PSI.

• Ordnance Survey should launch its proposed OpenSpace scheme, allowing non-commercial users free access to data, by December. The service is currently on hold, the review says, because smaller commercial users object to data being made available freely to potential competitors.

That’s got to be positive, surely.

“Power of Information” review from Cabinet Office: government could do more with our data

Thursday, June 7th, 2007

Tom Steinberg of MySociety (behind theyworkforyou) has co-authored an important new report, “The Power of Information Review” which suggests that online advice sites could improve citizen empowerment.

But it does much more: it looks at government use of public data and suggests more could be done, including beefing up the Office of Public Sector Information.

We’ll have more when we’ve read it in detail..

Tom Steinberg’s blogged his brief comments; here’s the official PDF; and here’s a site where you can comment on the report, rather as you can with Hansard discussion on theyworkforyou.

Meanwhile today’s Guardian looks at the collapse of the National Spatial Address Infrastructure effort in “Address plan finally abandoned“. RIP NSAI – you tried but didn’t have the right backing, ultimately.