Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens


Archive for April, 2007

Revealed: how profitable Royal Mail’s Postcode Address File (PAF) really is

Thursday, April 26th, 2007

The Royal Mail’s Postcode Address File (PAF) is pretty much indispensable for anyone doing direct marketing work. Now the Postcomm report on PAF that we mentioned last week has come up with something not seen before, at least outside RM: the profit and loss accounts for PAF. (Postcomm is the postal regulator, independent of RM.)

Turns out that PAF makes a profit of £1.58m on revenues of £18.36m, an 8.6% return on revenue. Most of the revenues come from PAF resellers (£14.9m). You’ll have to read to Annex 5 of the Postcomm report.

But we’ve looked at what this means in The Guardian. PAF underpins billions of pounds of the Royal Mail’s business, the direct marketing business, and the burgeoning market for satellite navigation (from memory, worth about £400m last year – corrections welcomed).

The price however is without any payment to local authorities, who have been cutting up rough about having no intellectual property rights embedded as suppliers of new address data. (There’s no information from Postcomm about how much PAF revenue comes from the public sector, which would be a useful figure. Royal Mail was chary about providing any information on PAF, one discovers on reading the report.)

RM told us it is “confident” about reaching a settlement. But isn’t this just more of a money-go-round, if they’re paid for supplying data to a product they then buy? Wouldn’t it make more sense to have PAF centrally funded, ringfenced by Postcomm’s recommendations and available for free to all? You might even encourage people to offer updates directly – which would cut the cost of maintenance, which at £9m or so comprises more than half the cost of running PAF for RM.

Postcomm makes recommendations on future of Postcode Address File: it should make a profit

Thursday, April 19th, 2007

Postcomm, the postal regulator, has come up with its recommendations on what should happen with the Postcode Address File (PAF) – that valuable item owned by the Post Office/Royal Mail.

Postcomm calls them “new safeguards for the future management of the postcode and address data contained in Royal Mail’s Postcode Address File”, which you can see at the Postcomm announcement on PAF:

The four key issues covered in today’s document – “Royal Mail’s future management of PAF” (pdf, 604KB)– are:

  • The definition of PAF – what information should Royal Mail be obliged to supply? Postcomm considers that ‘PAF data’ is not only made up of postcode details, but also includes other information needed to allow users to identify specific addresses.

  • The creation of an advisory board. Royal Mail has agreed to set up an advisory board to represent the views of PAF users, and has already started the recruitment process for the board’s independent chairman.
  • Ringfencing of PAF. As competition develops in the mail market – and also with other suppliers of similar address data – it is crucial that Royal Mail ringfences PAF from its other activities, in order to avoid potential conflicts of interest.
  • Profits. There is increasing demand for PAF data from a wide range of organisations, which rely very heavily on the information it provides. This puts Royal Mail in a very powerful position where setting prices is concerned. Although PAF does not fall within the ‘price control’ that Postcomm uses to set a pricing and service quality framework for Royal Mail, the company has agreed to aim for an operating profit margin in the range of 8-10%. If profits exceed this range, the excess would be either returned to customers or reinvested in PAF.
  • The linked PDF is a sprightly 92 pages, and I’m working my way through it. The questions that spring to mind for me are: why 8-10%? I suppose that’s a typical operating profit in the private sector – but it’s interesting that there’s not been the application of, say, “return on capital employed” (used in trading funds) which would give a much lower price for PAF.

    Secondly, how can PAF truly be ringfenced?

    We’re interested to hear your views – particularly if you’ve managed to reach page 92 first.

    Could free data have helped the Rural Payments Agency?

    Thursday, April 19th, 2007

    The Rural Payments Agency has quickly become famous, at least in Westminster circles, for losing £20m of our money and seeing its head fired for the failure which left farmers out of pocket.

    This week’s Technology Guardian looks at the extent to which bad mapping led to the problems, and asks whether free data – on the lines we’re advocating – could have averted it.

    The answer in truth is “not on its own”, because the RPA was a many-headed failure: the principal cause of problems was that the RPA decided to go beyond the European recommendations, and provide payments for parcels of land as small as 0.1 hectare (1,000 square metres) – one-third as large as required under the European Union rules.

    That followed all sorts of problems where maps were inaccurate or out of date, and where changes noted by farmers weren’t incorporated.

    Maps printed from the Land Register were sent to every farmer claiming subsidy to check. According to Julie Robinson, a lawyer with the National Farmers’ Union, this is where the system went wrong. “Many of the maps sent back to farmers to check turned out to be seriously inaccurate.” The maps missed land lost to floods, hedges and shadows from lines of trees. “It is all at the mercy of accurate mapping. The farmer depends on them to get it right.” The main problem, she says, was that the system was not matched to the needs of the users.

    Somehow we do feel that the Ordnance Survey and its Master Map would have coped better with the problem, as well as giving the Land Registry a huge boost in its attempts to find out who owns what parts of England. (Scotland and Wales coped better.)

    Could free mapping data have prevented the disaster? Probably not – mapping was only one factor in a complex mess of policy and management failures. But the fact that Defra was allowed to commission its own geographical database to an unworkably high specification suggests flaws in the government’s current way of working.

    There are two opportunities for change. One is a new geographical information strategy for the UK, now before ministers and expected to be published this summer. The second is the process of implementing the European Inspire directive, to create a “geospatial data infrastructure” across Europe. The lead department in transposing this directive into UK law is Defra. We suggest that when implementing Inspire it errs on the side of openness.

    In print: Canada’s maps go free – but here’s more background: it’s not so simple

    Thursday, April 12th, 2007

    Today’s Guardian has Canada drops licences and adopts free model for map data, which makes pretty much all the same points (possibly fewer, due to the limits of print space) as this previous post.

    Since writing it however I’ve also been contacted by Tracey Lauriault, of the Geomatics and Cartographic Research Centre, Department of Geography and Environmental Studies at Carleton University in Ottawa. She points out a number of things about the ‘free mapping’ movement in Canada, which I’ll quote at length since they’re all worth absorbing.

    The short form, though, is: Canada’s federal maps might be free, but the really useful data lie closer to the local level – and those are still charged for, quite substantially in some cases. Here’s what Tracey said by email, quoted with her permission (since sometimes people don’t want emails reprinted):

    I applaud what NRCan has done with its national framework data. Do keep in mind that NRCan topo data that was just made available is out of date and NRCan almost closed the office – read – this Hill Times article (subscription required for full text). We suspect they are making these data public and free to avoid having to continue the provision of paper maps. Canada is a country of wilderness lovers and tons of outdoors enthusiasts (canoers, campers, trekkers, hunters), forests, tundra, mines etc. where people go to very remote areas and well just cannot navigate using a blackberry screen or in remote areas where there is no satellite overage. See how the population is distributed along the US-Canada border to understand how critical it is to have up to date topo maps in rural and remote areas – which is most of the country!

    GeoBase (http://www.geobase.ca/) is very innovative indeed with how it is distributing national scale framework data and Geogratis (http://geogratis.cgdi.gc.ca/geogratis/en/index.html) distributes old, out-of-date free data. Both are part of the Canadian Geospatial Data Infrastructure. Both are going in the right direction; however, the good stuff we all want is not there. And both are setting the bar high for other Canadian organizations.

    Also read the Maps for Canadians site by Heather McAdam the GIS specialist for the MADGIC (Maps Data and Government Information Centre) at Carleton University. [There is more than one MADGIC in Canada – CA] Heather ran an excellent one woman campaign and mobilized our behind-the-times mapping associations in Canada. .. She is the reason for the topo map announcement.

    Canada is a huge country, with tons of geography and few people with three levels of government and division of powers – Federal, Provincial and Municipal. The feds have released the national framework maps, the shells of the national scale. The data to fill the shells such as health canada, statistics canada, environment canada etc. are by no means free or accessible or void of crown copyright. Also, to do any useful analysis at the local scale we need the provincial and municipal data sets which are harder to get – Manitoba being an exception. Other provinces sell the data at a very high price or with very restrictive use policies.

    Canada positions itself as being in between UK and US with cost recovery but not extreme like the UK. Statistics canada is very close to the UK model.

    The PR person for NRCan was very clever with their headline but this is not a huge breakthrough, just a nice press release, Geogratis and Geobase are far more interesting.

    Absolutely anyone can play and profit: Canada makes mapping data free for any use

    Wednesday, April 11th, 2007

    Canada has, as the geographers have probably noticed, made its mapping data free for download from the Geogratis site. You’ll need a fast connection and GIS tools to do anything useful with the data – but such tools are available for free all over the web.

    I held back on posting because I wanted to get confirmation directly from Natural Resources Canada (NRCan) – the ministry which looks after mapping inter alia responsibilities for the country’s natural resources, of which there are a lot – that this freedom extended to commercial enterprises wanting to build businesses around the data. (That isn’t clear from the announcement, nor from following the links around it: might commercial companies be restricted in their commercial reuse of the data?)

    But I’ve spoken to Ann Martin, who is director of the digital dissemination division at NRCan, and she confirms: yes, the data can be sold on without any royalties being due.

    That’s a change from the situation that used to prevail, where NRCan would license the data to users and resellers; there was also a royalty structure which meant resellers had to pay some of their earnings back to NRCan.

    Such a royalty structure is still in place in the UK: for instance, if you want to reuse a map from the London A-Z Map Company, about one-third of the charge is a royalty payable to Ordnance Survey. (OS knows about the Canadian move but has no comment on it.)

    Ms Martin told me that the previous licensing system was complex: “it almost cost more to administer than it brought in,” she said.

    However Canada’s charging system for maps was very different from that used by UK’s OS. Canada charged on a “cost of distribution” basis – that is, based on how much it cost to get the data to the customer, not a cost-plus recovery system like that used by OS where the charge is based on how much it costs to run the entire OS.

    That means NRCan brought in much less from selling its mapping – about C$400,000 (around £171,000), according to Martin. That’s a long way short of the OS’s £100 million – though as we’ve seen before, roughly half of that comes from within the public sector, meaning half of OS’s costs are funded indirectly by taxpayers.

    So, the difficult questions about this initiative:

    • how much is it expected to generate in new private-sector business, and hence taxes?
      Ms Martin doesn’t know – she says no formal study was carried out beforehand. But there has been interest from companies which had not previously shown any interest in the sector, apparently because of the licensing complexity.

  • How will the quality of Canada’s mapping be assured without any income stream?
    There isn’t an independent regulator; NRCan has an ongoing commitment to its mapping.
  • How much does it cost to collect the map data at present?
    Ms Martin doesn’t know. (The figures might be available somewhere, but she didn’t know where they would be.)
  • Will the Canadian government be strict in implementing its copyright over the maps and satellite data?
    Not particularly; users will be asked to acknowledge the original ownership of the data, and not distort or alter it.
  • There are key differences from the UK: Canada is much, much bigger, and its federal maps aren’t anywhere near as detailed as OS’s. The Geovisualisation blog has a very interesting take on this:

    I doubt Canada will ever be in a position to afford the highest resolution topographic data that will be needed in future applications. The country is too large. Thus, one strategy is to stimulate the private sector to build on the coarser resolution data. Consequently, Canada’s step to free-up topographic data is a step to ‘bridge’ needs against resouces.

    For, as Maps for Canadians notes,

    Canada’s maps are seriously out-of-date. In spite of daily landscape changes due to city growth and environment changes, Canada’s maps are on average, 27 years old. Canadians are concerned about security, safety and environmental health issues. Yet how can Government ensure critical services in these areas, if Canada lacks accurate and up-to-date maps?

    We’ve always recognised that free data is a two-edged sword; what you need is the right place for the mapping department to stand in so that everyone will benefit from the data being freely available. After the mess of the Rural Payments Agency, where mapping was key (though form-filling and processing also mattered), should Ordnance Survey be viewed as part of the Department of Environment, Food and Rural Affairs?

    Oh, and by the way: we’ve only just noticed this presentation (Powerpoint; or Googleised HTML) by Nancy Brodie, of the Canadian Treasury, from last year in which she notes the existence of the Free Our Data campaign. OK, so it worked for them. Could we have a few of the Canadian campaigners here too?

    Ed Parsons, formerly of Ordnance Survey, now of Google

    Sunday, April 8th, 2007

    Ed Parsons, formerly the chief technology officer until last December at Ordnance Survey (whose reason for leaving was never made clear), now has a new job and a

    new office…:

    “This week I joined Google as the Geospatial Technologist for EMEA (Europe, Middle East and Africa) and to say I am excited would be somewhat of an understatement.”

    This is intriguing news, really, for OS. The launch last week of Google’s My Maps (maps.google.com), which lets anyone create their own annotated maps, means that Google is encroaching more and more on territory that the OS could once – before the web – have called its own.

    Now, OS will still have a grip on low-level mapping down to the metre-accuracy level (companies wanting to dig up streets need to be sure where buildings are) for now.. but if Google really sees a market in something, that might not be the case for long. And it might also be true that Google is repurchasing OS data.. but not all of it.

    Things are changing. And this is quite a change. Will OS now go ahead with its OpenSurvey project, which Parsons tried to make happen?

    What if Ordnance Survey’s maps aren’t covered by copyright because they’re right?

    Thursday, April 5th, 2007

    A very intriguing story in today’s Technology Guardian, based on the analysis that you’ll find

    New study casts doubt on Ordnance Survey’s copyright control points out that

    According to a new study by government-funded intellectual property lawyers, some users at least have a legal right both to extract items of data and to pass them on to third parties. A study by Charlotte Waelde of the University of Edinburgh’s School of Law concludes that a geospatial database does not enjoy copyright protection, as Ordnance Survey claims, but rather is protected by the European Database Directive.

    What does that mean?

    Unlike copyright law, which can be used to block the reproduction of almost any part of a creative work – even John Cage’s 4’33” of silence – the database directive allows users to copy information, provided that it is not a “substantial” part of a database. The use must also be lawful and “not conflict with the normal exploitation of the database or unreasonably prejudice the legitimate interests of the maker”.

    You can find Mike Smith’s original posting on this.

    OS, as you might expect, OS sees it differently:

    “We haven’t been able to consider the report in detail,” said spokesman Scott Sinclair, “but there is absolutely no doubt that intellectual property rights exist in MasterMap – it would be ludicrous to suggest otherwise. In all our topographic information, there is copyright as in artistic works. Therefore use of those works without licence is an infringement.”

    Now, over to the lawyers…