Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens

Archive for the 'Government reports' Category

OPSI opens web channel where you can ask for government data

Thursday, September 20th, 2007

One of the fruits of the Free Our Data campaign’s meeting with Michael Wills, the minister in charge (inter alia) of the Office of Public Sector Information, is that we’ve been consulted on the creation of a web channel where people can ask for datasets from government.

That has now been set up.

As OPSI put it in the official press release (on the National Archives page),

The Office of Public Sector Information (OPSI), part of The National Archives, has launched a new online forum to engage with anyone interested in the re-use of government information for commercial benefit.

The channel will allow re-users of public sector information to request the release of government data that may have economic value.

The initiative is one of the recommendations from the Power Of Information report. We have met OPSI to advise on content, format and layout of the chanel, which can be found at – more specifically, in the Re-use request service forum.

In the welcome, John Sheridan of OPSI notes that the report says that

Recommendation 8 from The Power of Information says “To improve government’s responsiveness to demand for public sector information, by July 2008 OPSI should create a web-based channel to gather and assess requests for publication of public sector information.”

Good to see government beating a deadline.

The government has accepted this recommendation and OPSI would like to engage with the re-user community with setting this new web-channel up. This is why we have set up this discussion forum, to hear your views and discuss our plans.

So – what data would you like to see? Note of course that this doesn’t (necessarily) mean “free” data; there may well be a price on the data. But the question is, what do you think government has that you might want to see, and adapt, and turn into something? Never forget the power of mashing data – it could be that a few quite cheap data sources will reveal something very powerful.

(We also spotted and notified OPSI of a security hole in the way the forums had been set up – now fixed. So that’s two ways we’ve been useful.)

Ordnance Survey launches “OpenSpace” (sort of); Guardian reports on delays in government GIS report

Thursday, September 13th, 2007

Two related pieces of news.

First, the Ordnance Survey has launched – without any announcement we’ve spotted – an element of its OpenSpace platform, with the Explore portal. As originally conceived, this would have let people create mashups on OS maps. However, that functionality isn’t there yet; you can add “walks” (that seems to be all) to a 1:50,000 (Landranger-quality) map, though more is promised for the future.

Ed Parsons, formerly chief technology officer at OS, and now working for Google’s mapping division, comments on his blog:

Although this is nothing new – platial after all offered similar functionality a few years ago – this has been a long time coming. I was involved in some of the design work over a year ago! this is still an important step forward for the OS.

From a technology point of view the service was/is underpinned by the backend system developed to support the long delayed OpenSpace project, so hopefully there will be news about that soon.

Parsons concludes:

Although I would take issue with some of the T&C’s, this really is progress in the right direction from Southampton.

I’d not heard of Platial, but it certainly does do stuff that’s much the same (here’s a randomly-chosen “walk” in London), but using Google Maps.

The Explore page does show some Web 2.0-ness: it’s got a “blog” (more like comments) and shows the latest stuff. Except, as Parsons points out, any “walk” you submit becomes OS’s property. Eh?

Yes – here are the “portal rules“:

7.3 By submitting, posting or displaying any Submission on or through the OEP, User hereby grants to the Administrator a non-exclusive, perpetual, irrevocable, royalty free, worldwide licence to use, copy, edit, alter, reproduce, publish, distribute and/or sub-licence the whole and/or any part of the Submission, on or in connection with the OEP and for any other purpose.

In other words, OS can resell your stuff if everyone creates walks it likes. Not exclusively, but for itself.

Which in fact is exactly the same as we want the OS to provide its data to us, the citizens. Except that things seem to have gotten turned on their head, and OS is acting like a big media company such as MTV wanting to piggyback on the submissions of its viewers. (See “Whose content is it anyway?“, from Technology Guardian 21 September 2006)

Rather unhelpfully, although the page says that

The Administrator may update or revise the OEP Rules (including the Copyright / IPR Policy) at any time, with immediate effect, without notice. You are responsible for reviewing these pages regularly to ensure you are aware of any changes made and your continued use of the OEP after the changes have been posted means you agree to be legally bound by the new OEP Rules.

it doesn’t have a “last updated” tag.

Still, it’s movement, of sorts. But what we really want is an API so we can create mashups.

Update: Ed Parsons points out in the comments that it also bans links to “any page of the OEP Portal”. Let’s see how that one lasts online.

Meanwhile, Guardian Technology this week looks at the delays in the publication of an internal government report on geographical information:

Under the government-wide programme to transform public services through IT, a geographic information strategy for the UK was due to be published by July. But it has not yet appeared – and no publication date has yet been set.

Apart from the prime ministerial changeover, there is another reason for the delay: unhappiness that one organisation, Ordnance Survey, is both the government’s official adviser on geographical information and the main beneficiary of contracts to supply it. It’s roughly akin to Microsoft being appointed official adviser on government software purchasing.

The Association for Geographical Information has called for quicker publication of minutes of the government’s Geographic Information Panel; those from the June meeting haven’t appeared on the GIP’s website yet.

One final note of interest: this week’s edition of Guardian Technology carries a large recruitment advert on the front page – for staff in geographical information. It’s a burgeoning business…

Crown Copyright: time for an end?

Sunday, September 9th, 2007

In this week’s Guardian Technology we ask whether it’s time to end crown copyright. Time to take the jewels from the crown? examines the knotty question:

We argue that the UK government should follow the US in making all raw taxpayer-funded data available to the knowledge economy – except where that data compromises personal privacy or national security.

Some of our supporters say that a short cut to this state of affairs would be to abolish crown copyright itself. The idea is worth examining. Abolition was last floated in 1998, as part of a series of examinations in to what the government should do with its publishing arm, Her Majesty’s Stationery Office. A green paper, Crown Copyright in the Information Age (

/crown-copyright-in-the-information-age.pdf, 273KB PDF), proposed abolition as one of seven options for crown copyright.

In the public consultation that followed, abolition emerged as the most popular. From 70 responses received, abolition received 12 votes as preferred choice. The runner-up (retaining copyright but in a simpler form) received eight votes. The snag was that although abolition was the most popular response, it was also the least popular, receiving the largest number of “unacceptable” votes.

Faced with this polarised response, the government chose compromise. In 1999, the Cabinet Office found a “general consensus” in favour of retaining copyright, with simplified procedures for re-use.

There are arguments for and against crown copyright – certainly, the one that we hear from ministers is that it would mean you could be sure that something did originate where it claims to have done. (A hash on the original data could be followed through and computed on subsequent data to check its origin, for example.)

But the problem with crown copyright as it stands, and more importantly as it’s used, is that it’s used to restrict. That’s what copyright was used to do originally (over the Bible: see the introduction given at the RSA/Free Our Data debate – transcript, as PDF – last year by David Vaver of the Oxford Intellectual Property Group) and still is. Even the word itself has that ring. Maybe we need a different word.

Bonus link: Wikipedia’s page on crown copyright

Government seeks input on flooding review: got an opinion?

Wednesday, September 5th, 2007

The Cabinet Office has set up a “Flooding Lessons Learned Review” site, where you – yes, you, the citizen in front of the computer screen – can comment in a helpful way, one hopes, to prevent this summer’s floods repeating.

The terms of reference says that the “specific objectives” are (emphasis added here):

  1. To understand why the flooding was so extensive.
  2. To learn lessons on how in future we can best predict, prevent or mitigate the scale and impact of flooding incidents in a potentially changing environment.
  3. To look at how best to co-ordinate the response to flooding in future, including the significant social implications for communities.
  4. To establish what access to support, equipment, facilities and information is needed by those involved in the response at local, regional and national levels.
  5. To ensure the public has as much access as possible to information on the risk of flooding to allow them to take appropriate precautions, be adequately informed on developments as an emergency unfolds, and be looked after properly in the immediate aftermath.
  6. To establish how the transition from response to recovery is best managed.
  7. To identify those aspects of the response that worked well and should be promoted and reinforced.
  8. To make recommendations in each of these areas to improve the UK’s preparedness for flooding events in the future.
  9. To make recommendations, drawing on the experience of the flooding incidents, to improve the UK’s broader ability to manage the loss of essential services in any future emergencies.

We’ve already commented on how the Environment Agency restricts access to its flood data – a fact that is complicated, we now learn, by the fact that although the EA is a government-appointed agency, its data is not crown copyright (because it may have to sue government departments, which are “owned” by the Crown, and the monarch can’t sue him/herself. Follow that?) So not only would the Free Our Data campaign have to get trading funds reversed, it would have to get agencies paid by government to put their data under crown copyright. Honestly, it’s one step forward and one back.

Notwithstanding, it would make a lot more sense if the flood map data was simply available to everyone to use and even improve upon. We’ll suggest that in the flood review. You’re welcome to add your own comments on the Flooding Review site.

More on government departments’ submissions to the Select Committee

Thursday, August 2nd, 2007

This week’s Guardian Technology supplement expands on the acid remarks from Defra and also the Ministry of Defence over the Ordance Survey’s licensing system in submissions to the Communities and Local Government select committee inquiry.

In Ordnance Survey under fire from inside the government we note that

the Ministry of Defence paints a picture of a 200-year-old relationship turning sour because OS has to operate as a business. While the agency is a world-class organisation providing an excellent service, “in recent times the boundaries applied to the use of OS’s data for public service and national interest work have become increasingly blurred. MoD has experienced more stringency and complexity being applied to the release of data by OS, which has resulted in uncertainty and lack of flexibility in the use of that data by the MoD.” Licensing charges set by OS are “particularly high”. As a result “some government users are being denied access”.

Sound familiar?

Also worth noting: there’s now a single page with all the Free Our Data stories from the Guardian. (It’s part of the website redesign. Maybe we’ll try on here some time.)

Government asks Free Our Data to work with OPSI on web channel for users

Thursday, July 26th, 2007

We did say that the meeting with Michael Wills was interesting. In today’s Guardian, we precis the meeting (update: the full text is now on the blog), which boils down to a few key points:

  • Michael Wills, the minister for information at the Department for Justice, thinks the case for free re-use of public sector data in and outside government is “compelling”
  • He thinks that government has to move quickly to adjust to the changing information world
  • He expects the ‘public task’ of Ordnance Survey to be defined within a study that will report to him by the end of this year
  • the trading fund model, set up in the 1970s (by which government agencies charge for their output, reducing their need for direct tax funding) needs reexamination
  • He is reluctant, however, to disturb a model (trading funds) which has produced high-quiality information

They key point though is that he wants Free Our Data to work with the Office of Public Sector information to set up a web channel through which the public can request public data, and what form they want it. That, to us, marks a significant recognition of the importance of this campaign.

Read more at The minister will hear you now in the Guardian. The full transcript of the meeting will be posted once it has been re-checked for accuracy.

We’re going to see the minister..

Saturday, July 14th, 2007

Perhaps the Free Our Data concept is getting some consideration in ministers’ minds. One of the first thing that Michael Wills did on being promoted to his new job at the Department of Justice was to ring us up and ask for a meeting.

Eh? Ministers calling journalists? Quite a turnaround. Not even his private office, but the man himself.

As a result we’re going to see him later this week, to make our case formally – and, perhaps, to see if there’s any formal response beyond that on the OFT’s Commercial Use of Public Information report, and perhaps the Cabinet Office response on the Power of Information.

Among the examples we’re planning to use re government making things free providing a boost to commercial sectors:

  • GPS: the US government-funded Global Positioning System costs about $400m to run per year, but underpins a huge business reliant on its positioning data
  • museums: the Labour administration made admission to museums free. Why? To encourage more people to go to them. (The Conservatives propose to reverse it.. perhaps) It has led to more visits (though Hazel Blears points out that it’s not all been encouraging .. but a letter in June from senior museum directors says they like it – but that it needs funding. (Has anyone analysed the economic effects of 30m extra visits to museums – as in the train and bus journeys, the gifts bought, and so on?)
  • the examples of South Africa, Canada and New Zealand in making at least some of their data free

Anyone else with the sort of arguments that would help a minister persuade the Treasury to unzip the purse strings and give the information economy an extra boost? Your suggestions – with sources, please – welcome.

Europe’s Galileo satellite program faces more obstacles

Wednesday, July 11th, 2007

Following the debate in the House of Commons recently, in which it was pointed out – or at least alleged – that the US’s Global Positioning System (including a useful potted history) now underpins the entire US economy, but that nobody could come up with a convincing reason why we need Galileo too.

Owen Paterson, for the Tories, said:

Last week, I had meetings with representatives of Trafficmaster plc, a highly successful company selling navigation services to more than 100,000 vehicles in the UK. Its technical director, Christopher Barnes, said that

“the free to air GPS service is sufficient for vehicle navigation and therefore we are unlikely to be interested in paying (either voluntarily or through a compulsory tax) to use a European service, even if technically it would be better.”

There is extremely limited application for the higher accuracy that Galileo will offer and, in any event, any such advantage will last only until the US deploys Block III Navstar, which promises equivalence.

(That’s an interesting statistic: 100,000 vehicles using TrafficMaster. And that’s only one of the many satnav systems on sale. How much taxable revenue does GPS – a free government data service – generate?)

Now the Guardian notes that Galileo faces further obstacles: in “Funding row pushes GPS system further off course“, it quotes Olivier Houssin, head of the commercial and security operations of French electronics group Thales, saying Europe runs the risk of being left behind in key commercial and military applications by the US, China, Russia and India if it doesn’t back Galileo.

“If Galileo collapses it will be the collapse of the most important EU programme outside the common agricultural policy,” he said in an interview. “Europe is stagnating in space.” EU transport ministers agreed last month to scrap the public private partnership for building and running the 30-satellite Galileo system, which promises greater accuracy than the American GPS, to control air and road traffic. It will also provide enhanced civil security and even help to pilot driverless trains. Mr Houssin dismissed the PPP/PFI as “a false good idea” because Galileo was a “strategic infrastructure” that Europe had to fund publicly. He said the French and Germans were now at loggerheads over how to provide the extra €2.4bn required.

Berlin favours making extra voluntary contributions to the European Space Agency, which is in charge of the overall project, in return for a greater share of the workload. “They want to take over the technological leadership of the programme and centre it around the activities of Astrium [the space arm of EADS] in Munich,” he said.

The French would prefer to see the project financed as an EU investment, with cash sourced from other European commission budgets.

Britain, which still clings to the notion of a PPP/PFI, has refused to give the go-ahead for switching entirely to public funds.

It is very difficult, as Owen Paterson pointed out, to find a way in which Galileo is necessary when GPS III is on the way – unless, that is, its real value lies in military application. But that supposes us not being an ally of the US, which is a very peculiar worldview to start from.

(Thanks to Rob for pointing to the debate.)

Catching up: government responds to OFT and Power of Information reports

Thursday, July 5th, 2007

Should have posted this earlier, but I was wondering whether the disappearance of the DTI would kill the links. (It hasn’t – still works, but just has a different name. By their web addresses ye shall know them…)

Anyway, on the last Monday of June the government finally replied to the OFT “CUPI” report on the Commercial Use of Public Information, and to Tom Steinberg’s and Ed May’s Power Of Information report.

The response to CUPI is from the DTI, or as it’s now known the Dept for Business, Enterprise and Regulatory Reform. The PDF response is at (78kB).

The Cabinet Office response to the Power Of Information is at a page titled “Power Of Information principles get go-ahead from Government.”

There’s also a “Have your say” page attached to the Cabinet Office page; see what you think.

The key question though: will trading funds be forced to consider themselves? Yes. The government is asking for an investigation into the cost benefit justification for organisations working as trading funds.

We reported this as “Government on the back foot over policies for pricing data“:

The long-awaited reports, from the Department of Trade and Industry and the Cabinet Office, recommend that the government make more data available without strings to community and commercial ventures. The Cabinet Office also paves the way to government sites opening self-help forums for citizens, and civil servants engaging openly in independent forums, blogs and wikis.

Both reports, however, avoid the central demand of Technology Guardian’s Free Our Data campaign – that government should stop running information businesses, and instead give all data away to stimulate the knowledge economy.

On the plus side,

Notably, it urges Ordnance Survey to launch its postponed OpenSpace project, which would provide a Google Maps-like interface for mashups, by the end of December.


However, the response warns that more ambitious steps towards free data could threaten the business of trading funds and their “vital role in the UK economy”. It agrees with the Power of Information review, which says that the government should get hard data from an independent study of the costs and benefits of the trading fund model.

So we’ll ask for an on-the-record ministerial interview once we’ve figured out who, under the new regime, is responsible for Cabinet Office, trading funds and public sector information…

Minister confirms government response to OFT on PSI by end of June

Thursday, June 14th, 2007

Wondderful thing, theyworkforyou – it takes government (well, Parliamentary) data and repurposes it to create something more useful.

We’ve just noticed that among the written questions is one by Mark Todd, who wanted to know when the government will respond to the OFTs’ CUPI report.

The response of Ian McCartney, DTI minister:

The OFT’s report contains some challenging recommendations. Given the importance and potential impact of the recommendations, and the wider constituency of interested parties both within and outside Government, more time was required to properly frame the response. The Government expect to publish their response to the report before the end of June.

We’re intrigued by the idea that there are “challenging” recommendations in the OFT report. But at least this gives us a clear indication that things are moving.

“Power of Information” review from Cabinet Office: government could do more with our data

Thursday, June 7th, 2007

Tom Steinberg of MySociety (behind theyworkforyou) has co-authored an important new report, “The Power of Information Review” which suggests that online advice sites could improve citizen empowerment.

But it does much more: it looks at government use of public data and suggests more could be done, including beefing up the Office of Public Sector Information.

We’ll have more when we’ve read it in detail..

Tom Steinberg’s blogged his brief comments; here’s the official PDF; and here’s a site where you can comment on the report, rather as you can with Hansard discussion on theyworkforyou.

Meanwhile today’s Guardian looks at the collapse of the National Spatial Address Infrastructure effort in “Address plan finally abandoned“. RIP NSAI – you tried but didn’t have the right backing, ultimately.

APPSI comes out in favour of Ordnance Survey on addressing – but it’s two-edged

Thursday, May 3rd, 2007

Deep waters here: this is a case where what goes on in public is more subtle than at first appears. Read on, and you’ll find – we think – that the government is being forced to define precisely where the Ordnance Survey’s “national obligation” ends and its “commercial” (that is, optional, non-core) activities begin…

Today in The Guardian we report on how the Advisory Panel for Public Sector Information has determined in favour of Ordnance Survey in the row between OS and Intelligent Addressing, which runs the National Land and Property Gazeteer (NLPG).

The APPSI said that it couldn’t really rule on the matter – which drew what could be seen as an affronted response from the Office of Public Sector Information, which said that APPSI was making a “literal interpretation” of the rules governing PSI.

As the article explains,

Intelligent Addressing, which operates a gazetteer compiled and run by local councils, complained in February 2006 about the way Ordnance Survey licenses its address database, called AddressPoint.

Intelligent Addressing complained to OPSI, formerly Her Majesty’s Stationery Office, which oversees two compliance mechanisms: the public sector information regulations and a “fair trader” scheme. In July, OPSI’s report backed some of the firm’s complaints. Both sides then asked the APPSI, an expert group responsible to the Department for Constitutional Affairs, to review the findings.

In a 17-page report published on Monday, the advisory panel says that Ordnance Survey’s AddressPoint product is not part of the mapping agency’s “public task”. As such, it cannot breach regulations covering the supply of public sector information. The APPSI recommends that the company take any further complaints to the Office of Fair Trading (OFT).

Here’s the interesting point. If AddressPoint isn’t something that OS needs to do (because it’s not “public task”), then that must be something that lies on the commercial, not PSI, side of its operations. In which case there must be other data that needs to be defined as being the public task.

But equally, one would expect that like the Met Office, which has to charge itself fairly for the data it collects and then resells, this will prevent the OS cross-subsidising itself. In effect, it would strip back what the OS does to a “public task” bone.

Richard Susskind, the head of APPSI, may have made a clever move by palming this off. It in effect forces OPSI to determine what OS’s public task is, and what the limits of PSI are.

We’ll watch with interest.

Revealed: how profitable Royal Mail’s Postcode Address File (PAF) really is

Thursday, April 26th, 2007

The Royal Mail’s Postcode Address File (PAF) is pretty much indispensable for anyone doing direct marketing work. Now the Postcomm report on PAF that we mentioned last week has come up with something not seen before, at least outside RM: the profit and loss accounts for PAF. (Postcomm is the postal regulator, independent of RM.)

Turns out that PAF makes a profit of £1.58m on revenues of £18.36m, an 8.6% return on revenue. Most of the revenues come from PAF resellers (£14.9m). You’ll have to read to Annex 5 of the Postcomm report.

But we’ve looked at what this means in The Guardian. PAF underpins billions of pounds of the Royal Mail’s business, the direct marketing business, and the burgeoning market for satellite navigation (from memory, worth about £400m last year – corrections welcomed).

The price however is without any payment to local authorities, who have been cutting up rough about having no intellectual property rights embedded as suppliers of new address data. (There’s no information from Postcomm about how much PAF revenue comes from the public sector, which would be a useful figure. Royal Mail was chary about providing any information on PAF, one discovers on reading the report.)

RM told us it is “confident” about reaching a settlement. But isn’t this just more of a money-go-round, if they’re paid for supplying data to a product they then buy? Wouldn’t it make more sense to have PAF centrally funded, ringfenced by Postcomm’s recommendations and available for free to all? You might even encourage people to offer updates directly – which would cut the cost of maintenance, which at £9m or so comprises more than half the cost of running PAF for RM.

Postcomm makes recommendations on future of Postcode Address File: it should make a profit

Thursday, April 19th, 2007

Postcomm, the postal regulator, has come up with its recommendations on what should happen with the Postcode Address File (PAF) – that valuable item owned by the Post Office/Royal Mail.

Postcomm calls them “new safeguards for the future management of the postcode and address data contained in Royal Mail’s Postcode Address File”, which you can see at the Postcomm announcement on PAF:

The four key issues covered in today’s document – “Royal Mail’s future management of PAF” (pdf, 604KB)– are:

  • The definition of PAF – what information should Royal Mail be obliged to supply? Postcomm considers that ‘PAF data’ is not only made up of postcode details, but also includes other information needed to allow users to identify specific addresses.

  • The creation of an advisory board. Royal Mail has agreed to set up an advisory board to represent the views of PAF users, and has already started the recruitment process for the board’s independent chairman.
  • Ringfencing of PAF. As competition develops in the mail market – and also with other suppliers of similar address data – it is crucial that Royal Mail ringfences PAF from its other activities, in order to avoid potential conflicts of interest.
  • Profits. There is increasing demand for PAF data from a wide range of organisations, which rely very heavily on the information it provides. This puts Royal Mail in a very powerful position where setting prices is concerned. Although PAF does not fall within the ‘price control’ that Postcomm uses to set a pricing and service quality framework for Royal Mail, the company has agreed to aim for an operating profit margin in the range of 8-10%. If profits exceed this range, the excess would be either returned to customers or reinvested in PAF.
  • The linked PDF is a sprightly 92 pages, and I’m working my way through it. The questions that spring to mind for me are: why 8-10%? I suppose that’s a typical operating profit in the private sector – but it’s interesting that there’s not been the application of, say, “return on capital employed” (used in trading funds) which would give a much lower price for PAF.

    Secondly, how can PAF truly be ringfenced?

    We’re interested to hear your views – particularly if you’ve managed to reach page 92 first.

    Free our address data – or at least get them to stop charging each other: the petition

    Tuesday, March 13th, 2007

    As noted in this comment elsewhere on the blog, there’s now a No.10 petition to stop the address madness:

    We the undersigned petition the Prime Minister to stop the expensive and damaging address ownership conflict currently existing between Royal Mail, Ordnance Survey and Local Government by definitively establishing that the intellectual property input of all three parties into derived address products such as PAF, AddressPoint, NLPG and NSG is equal in scope and value. Consequently, in the public interest, none of the three parties should charge either of the other parties for use of these products.

    While we’re sceptical of the power of these petitions to change anything, they’re a good meeting point to indicate strengths of feeling.

    So – sign it and then meet back here to work out what financial impact this would actually have – especially since local authorities are meant to be planning how much to charge for searches relating to homes (link to Yvette Cooper MP dithering on the matter earlier this month).

    If they didn’t have to pay for that part, would searches be cheaper?