Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens

Archive for the 'Financial analysis' Category

OS publishes almost unredacted version of international study

Sunday, July 26th, 2009

The OS has at last, in response to my FOI request (thank you published an almost unredacted version of its international study, fully known as ” International Comparison of Geographical Information Trading Models – Study report”.

You can download it from the OS’s page about it, which contains the interesting addition (or is it that we only just noticed it?) that this is a study “which was commissioned by the Parliamentary Under Secretary of State for Communities and Local Government, Iain Wright MP, and formed one input to the Trading Funds Assessment undertaken by the Shareholder Executive and HM Treasury.”


Anyhow, your opinions on what the study tells us – and especially whether it actually does manage to confirm any of the things that it was trying to confirm – are extremely welcome. Comments as ever are open.

That international consultant and man of mystery is… (updated)

Thursday, July 23rd, 2009

…is still a mystery. Ordnance Survey today responded at 18.18 BST (by my computer’s timestamp) to my request for the name of the international consultant who looked over the OS’s study justifying its own findings.

And the outcome:

I can confirm that Ordnance Survey does hold this information

I suppose it might have been done by anonymous peer review, but it’s unlikely..

however I regret to inform you that your request falls within the ‘Personal information’ exemption under section 40 (2) (a) of the Freedom of Information Act 2000 (FOIA). We believe this exemption applies because having contacted the person concerned Ordnance Survey has been requested to withhold their name in this instance, therefore we will not be releasing this information to you.

This person is turning in to some kind of remarkable beast. Let’s just remind ourselves of what we wanted to know: who is it that OS described as an “internationally recognised expert in geographical information and national mapping” who OS said had reviewed its study into how it should organise and charge for its service, and who “agreed with the analysis and conclusions”.

And now this person turns out to be such a shrinking violet that they don’t want to have their names in the papers and on the web?

This “internationally recognised expert” doesn’t want to be named? Doesn’t want to be named on a strategic study into one of the most important mapping agencies in the world?

This is, quite frankly, ridiculous.

We will now begin making representations to the minister ostensibly in charge of OS, Iain Wright.

And we haven’t even looked at the latest version of the study itself that arrived by email. Wonder what non-gems are buried in that.

Meanwhile, the search for Spartacus goes on…

Update: First: OS points out that since a recent reshuffle, Iain Wright is no longer the minister responsible; it’s now Ian Austin.

Second: following comments, I have put the question directly to Robin McLaren of Know Edge (mentioned in the comments below as a possible International Consultant and Man of Mystery). His reply, besides that in the comments below where he says Know Edge was not retained, is:

No, I have not been involved in the creation or review of the Ordnance Survey study. Like you, I am mystified as to who this ‘International Consultant’ may be.

In the Guardian: one-off census address database cost rises 20% to 12m pounds

Monday, March 9th, 2009

In The Guardian, we’ve followed up on the question put by the Tories’ social affairs minister (if I recall correctly) Eric Pickles, who asked about the price of the one-off census address database for 2011.

It turns out it has risen from £10 million to £12m in the space of just a few short months. Hasn’t anyone told them there’s a deflationary recession on?

As a reminder: the need for this database has grown from the fact that the Ordnance Survey, Royal Mail and local authorities can’t agree on how to build an address database that the Office for National Statistics can agree will be definitive for carrying out the 2011 census.

Well, fair enough, you might say. The ONS’s requirements differ subtly from OS, RM and local authorities. It’s possible that their interests and plans won’t be entirely congruent.

What’s mind-boggling, and completely idiotic, is that ONS is going to build this database for itself (must already be in the process, since it doesn’t have long to do it) and then is going to destroy it. Because nobody could agree to let such a thing exist independently.

It’s an egregrious waste of money – first the building, which is madness, given that the data already exists; and then the destruction. Occam’s Razor of datasets: don’t let entities multiply needlessly. And the first law of not wasting things: don’t destroy hard-won datasets needlessly.

The “Jersey question”: what if the profits of free data move offshore?

Monday, November 3rd, 2008

I gave a talk last month about the Free Our Data campaign to be2camp (which had the aim of getting the idea of better sharing of data through wikis and other social systems for architecture).

Amidst it there were questions, and among the questions was this one: “if you make all the data available for free download, what’s to stop companies from relocating to Jersey – which pays no UK tax – and operating from there?”

That would mean that the UK taxpayer now wouldn’t be getting the benefit of revenues to the Ordnance Survey through its licensing fees, and wouldn’t get the tax revenue on the company that would now be headquartered in Jersey. So you’d have higher taxation costs (since the OS would now be funded centrally) and no obvious economic benefit.

So what’s the answer?

We don’t know.

The government has struggled with this problem over issues such as gambling, where companies have preferred to locate in places like Gibraltar or far-flung Caribbean countries with less strict tax regimes and offer services online. There’s a money flow out of the UK with those too.

One possible argument is that even if the data were sold and the profits taken abroad, the UK economy would benefit because the information is being made more widely available – and that has to have a benefit. There could even be a multiplier effect, as there tends to be with any commodity: making steel bars is a profitable business (or can be), but making buildings is a much more profitable one. Perhaps in time, with a free data model, the OS data would be the steel bars of the building: necessary, but not the biggest part of the value chain.

The postcode debate, summed up beautifully on Tom Watson’s blog

Monday, July 7th, 2008

Tom Watson MP, the Cabinet Office minister who is also the political wing of the Power Of Information taskforce, started an interesting debate on his blog, when he noted a comment by Simon Dickson about the usefulness of the Postcode Address File.

He mused, “I’m going to spend some time trying to understand just why [PAF] can’t be available for free or at marginal cost. Feel free to air your views in the comment section.”

And boy, did people air their comments. It’s worth reading in full, but I think the prize – at least the Free Our Data prize for stating the value of the free data model – goes to Greg, who (in a long and well-argued comment) sums up by responding to “Mitch” (an earlier commenter who had worked in the Royal Mail on updating the PAF):

The points you make, Mitch, are unfortunately so reminisecent of the innovation-stifling opinions of inward-facing bureaucrats which have been such a major contributor to Britain’s loss of economic advantage over the years. Examples which are now so clear include the fact that we invented public-key encryption long before the US, but kept it a government secret rather than using it to gain an edge in commerce; or that Frank Whittle invented the jet engine only to find that closed-minded bureaucrats couldn’t see it working. Bureaucrats are rarely the best people to judge whether something has a place in propelling innovation and competitiveness. The fact that there’s so much energy on my side of the postcodes debate [arguing to make it available for free] says it all.

Mitch; you should be proud that you worked on a world-leading data source. It’s just such a shame its wings are crippled by its owners.

We love it when people state the benefits so clearly. The whole thread is worth reading, though, for the vigour of the arguments on both sides.

Trading Funds review: terms of reference

Wednesday, June 25th, 2008

Things are starting to happen, through the work of the Power of Information taskforce.

Here’s the latest: the terms of reference for the study demanded by the POI report reviewing the trading fund model have just been published.

The study will be jointly led by Treasury and DBERR; Yvette Cooper and Baroness (Shriti) Vadera are the lead ministers, though the Cabinet Office POI taskforce secretariat will work closely with them. (We’re still waiting on the timescale.)

From the POI Taskforce blog:

the Taskforce will be assisting with this review, particularly looking at the value of the data held by the funds and whether the current business models and licencing arrangements are sustainable.

If you had to set our priorities what areas would you have us look at? Is there any research we should be looking at? Please kick off the discussion in the comments.

From the TORs themselves, part of the detailed work will include

With regards to a priority group of TFs (Ordnance Survey, Met Office, UK Hydrographic Office, Companies House, DVLA, and Land Registry) the exercise will aim to produce a detailed and definitive pricing and access policy for information held/created by TFs and the optimal constitutional structure for the several TFs to maximise benefit to the UK economy whilst maintaining public policy objectives.


changes in the pricing policy and licensing regime around information held by TFs may impact on their trading performance and value. Therefore for those TFs affected, the assessment will
  • provide an assessment of the business model or constitutional reforms needed to meet the Government’s commitment on access to information collected for public purposes by downstream markets; and
  • provide an assessment of the options for pricing the release of this information

We think this is going to be fun.

Could a surcharge on planning applications fund free data from Ordnance Survey?

Tuesday, April 1st, 2008

Since we’re playing with hypothecated taxes – which we should point out is not a central tenet of the Free Our Data campaign; we think governmnent should fund this centrally, but recognise that in tight economic times it’s hard to persuade government to spend more rather than doing revenue-neutral “experiments” – let’s examine the suggestion (made elsewhere) that OS’s free data should be funded by a surcharge on planning applications.

After all, the logic goes, planning applications are voluntary; and they require OS detail. (One has to submit a number of copies of OS maps of the area the application applies to.)

[A disclosure up front: I have a planning application in process. However, the following simply emerges directly from the available statistics.]

Let’s recall the amount that needs to be raised, according to the Cambridge study, to compensate OS for the loss of revenues from selling its “raw” or “public task” data: £12m – £30m.

Now, how many planning applications were there for the UK? For England and Wales, the total applications received looks like this:

This is only for England and Wales, of course, but shows that planning applications, even at their busiest, are an order of magnitude less than Land Registry transactions. True, this excludes Scotland, but even if that has as many planning applications as England and Wales (which seems unlikely), that would only be a total of about 1.3m applications in the busiest time.

That in turn would imply a surcharge per application received of roughly £10 to £30. Whether that’s small or large compared to the cost of the overall transaction (which might be putting up an extension on a house or a change in use, requiring no actual building) isn’t clear.

(Locally, making an application costs between £135 – £265. It may vary in other locations. The surcharge would also have to be collected from multiple councils, which would impose an administrative overhead; by comparison, there’s only one Land Registry.)

But as an actual amount per transaction, it’s comparatively large, meaning it would be very sensitive to variations in the number of applications. The figures there vary from 501,000 to 689,000 – a 28% or 37% variation, depending which you take as your numerator.

Why might this be? Perhaps Land Registry transactions include commercial purchases, which occur more frequently. And also that people buy and sell houses more frequently than they do things to them. Either way, it’s doesn’t look like the optimum way forward. It’s certainly a lot more expensive – per transaction – than a Land Reggistry surcharge.

Year or Quarter

received (,000)





















* provisional. Source: DCLG

Land Registry surcharge could fund free OS data surprisingly cheaply

Thursday, March 27th, 2008

Sold signs outside houseOne suggestion that has been made by Robert Barr (of Manchester Geomatics) and echoed recently by Ed Parsons on his blog (though I think Ed came up with it independently) is that Ordnance Survey’s non-refined data (that is, the stuff it does as part of its public task, which the Cambridge economics study of trading funds interpreted to be its MasterMap and Large Scale Topo) could be made available for free by making up any funding shortfall from a surcharge on Land Registry transactions.

The reasoning: most LR transactions involve OS mapping.

According to the study, that would cost between £12m and £30m in foregone revenue.

So how much would you have to add to Land Registry transactions to make up that amount? It sounds like an awful lot of money to generate.

Here are the figures I’ve culled from the Land Registry’s performance data for the past three years on the number of transactions.

Number of registrations 2004/5 2005/6 2006/7 Mean 04-06 As % of total
first registrations 297,405 309,609 304,391 303,802 4.3
discharges 2,486,875 2,502,318 2,605,620 2,531,604 35.8
mortgages 2,680,128 2,627,999 2,723,530 2,677,219 37.9
transfers for value 1,378,200 1,270,867 1,480,819 1,376,629 19.5
leases 167,234 173,610 197,546 179,463 2.5
Total 7,009,842 6,884,403 7,311,906 7,068,717 100
Total w/o discharges 4,522,967 4,382,085 4,706,286 4,537,113 64.2

With millions of transactions, it looks like raising £12m – £30m wouldn’t actually be too hard. “Discharges” are the ending of a claim to a legal title – generally, though not always, the end of a mortgage. They attract no fee at present. Other LR charges range from £2 (for a search) to £700 (for first non-voluntary registration of a pricey parcel of land). Most of the charges, though, are £20 – £40 and upwards.

So to find the £12m that the trading funds report suggests OS would lose solely from non-discharge transactions would mean adding £2.65 to the cost of each LR transaction.

If we take the loss in revenue to OS as £30m, then it means adding £6.61 to each transaction. It’s not more than the cost of any transaction (except searches – which aren’t the same as the “searches” one does when buying a house; those go through your local authority), and compared to the cost of the typical transaction – say, the average £180,000 house purchase – it’s peanuts.

Right – that’s the analysis done. Now we just need to find a minister who is in charge of Land Registry and Ordnance Survey and can tweak the legislation (it doesn’t need primary legislation, surely?) to make these changes. And we’re done.

This analysis also appears (without the fun table) in today’s Guardian: Land Registry holds key to free OS.

Trading Funds report: final totals: economy +£179m, gov’t -15.4m

Wednesday, March 19th, 2008

So here are the tallies, according to the Trading Funds report, of how making bulk data free would benefit the economy, and how much it would cost the government. (You can find the figures in Chapter 7 of the report.)

Name Product Gross benefit (£m) Net cost to government (£m) Net gain (£m) Ratio of return on investment (net gain/net cost)
Ordnance Survey Large Scale Topo 168 12 156 13
DVLA Anonymised Bulk & Mileage data 4.3 0.582 3.7 6.36
Companies House Bulk Data and Image 2.6 0.681 1.9 2.79
Land Registry Property Price, Polygon GIS 2.3 1.1 1.2 1.09
Met Office wholesale data 1.2 0.260 1.03 3.96
UK Hydrographic Office Digital UK Charts & Publications 1.08 0.744 0.338 0.45
Total 179.48m 15.37m 164.2m 10.68
source: models of PSI provision by Trading Funds report, DBERR

The figure for Ordnance Survey is dramatic, but understandable: as it says itself, it underpins huge amounts of economic activity. The point that this study makes is that making its raw data free would allow even more economic activity – creative, useful, beneficial, taxable – to occur.

(The UKHO is unlike the other trading funds, as the report points out, because it takes in raw data from outside organisations, which it may not be allowed to make available royalty- or payment-free. Which goes some way to explaining the answer to the question I was asked earlier.)

Advance notice: Mike Cross talking to BCS on April 2 in London

Wednesday, March 19th, 2008

Just in case you’ve got a slot in your diary: Michael Cross, the co-founder of the Free Our Data campaign, will be talking at the British Computer Society’s Geospatial Specialist Group on Wednesday April 2 at 6.20pm in London.

As the page says, “Michael will be explaining the rationale behind, and taking questions about, the sometimes controversial campaign, now in its third year.”

Better to be sometimes controversial than always bland, though, that’s what we say.

The page says that “NB This event is free, but registration is required. If you would like to attend this event please register by sending an email to”. It’s not clear if you need to be a member of the BCS or its Geospatial Specialist Group to come along – but presumably the registration email will tell you.

We suspect that Mike will talk a little about the strength of the official reports that have all, so far, backed more or less the campaign’s standpoint; none has demonstrated that the wider economy would lose out by adopting our system.

Ordnance Survey reviewing paper map licences – with a view to removing them?

Thursday, October 18th, 2007

You think you’ve got something good going, and then it gets pulled away.

We’ve seen a letter from the OS to someone considering a paper map licence – which, as noted earlier, can save thousands of pounds over the “web-based” licences, by allowing you to put scans of maps online.

At the time we queried whether the Paper Map Copying Licence (PMCL), which costs about £50 per year, could possibly be cost-efficient, since it would surely cost more than that just to administer.

Possibly someone heard us in OS. The latter from its licensing department reads, in part (emphasis added, by us):

We should let you know that the PMCL is currently undergoing an internal review. It was originally created a number of years ago and the intention behind the display and promotion rights was only to permit some very limited uses, such as a business showing their location to customers. With time, we understand that these rights could be interpreted more widely than was intended and overlap with other ways in which we license businesses to use our data.

Should you now choose to take out a PMCL, it will remain valid under those terms (subject to termination in accordance with its provisions) for 12 months. After that time, depending on the outcome of our review, we may need to inform you that other licence terms will cover your use. Of course, we will give you as much notice of this as possible.

Ordnance Survey does endeavour to achieve consistency in its licensing terms so that comparable uses are licensed on comparable terms. However, the number of uses to which our data are put – which are continually growing – sometimes means that there is unintended and unanticipated overlap between licences.

Translation: this seems to be a loophole, so we’re going to try to close it. Lots of administrative effort and pencil-sucking meetings and civil service – sorry, trading fund – biscuits will be expended on trying to find legal wordings which preclude you from putting stuff on the web (where so much of business now happens) without paying us lots of cash.

I’m giving a presentation to EPSI Plus next week, about the Free Our Data campaign. I’m toying with the idea of showing how Google would look if it implemented OS licensing on IP.

Information World Review points to benefits of “fair use’ on economy

Thursday, September 20th, 2007

Information World Review, which has joined the Free Our Data campaign, has pointed to a study comparing the economic benefits of industries which use copyright-controlled data, and those which use non-controlled data.

The copyright-controlled side generates less economic benefit under the study (though we have to admit having seen a critique by Nick Carr, a columnist for Guardian Technology, pointing out that the study is amazingly wide-ranging – and includes entire industries which may lie on both sides of the “copyright/unrestricted” fence.

It is difficult to do economic analyses of such things. For instance Google runs on the free Linux operating system. But you won’t find it giving out copies to Microsoft (even if Microsoft wanted it). Is that copyrighting, or unpaid use? The argument surely is that Linux *enables* Google to build a huge business which relies, internally, on all sorts of copyrighted, and closely-guarded details. In the same way, making government data free for the benefit of all citizens would give us the foundations for new businesses which could do remarkable things. GPS remains the classic example.

Want to put maps online cheaply? Get a paper licence and scan them

Thursday, August 30th, 2007

This will seem really strange, but tipped off by a comment on this blog from Walkhighlands, we investigated further. And it turns out to be true: with an Ordnance Survey “paper licence”, you can scan maps and put them online as you like.

Price difference compared to the digital licence: it’s about 1/50th price, or more if the digital licence waas quoted by a reseller.

There’s more at Paper maps rather than digital ones save site 99% in OS fees, which looks at the problems that Walkhighlands – an enterprising new small business based in Skye, which offers walks in those most beautiful (and remote, and in need of urban visitors looking to spend some time and cash, such as the picture above of Skye) – had in trying to put digital versions of the maps of its walks online.

Set up only this February, it operates from Staffin, in northern Skye, and already gets an average of 600,000 hits every month from about 18,000 visitors a month. It has recently been chosen as one of 26 new businesses to receive funding from Nesta, the National Endowment for Science, Technology and the Arts, and features more than 250 walking routes, which its organisers are expanding all the time.

But when Paul Webster, who runs the site, inquired into the costs of putting printable Ordnance Survey maps on the site (which already offers links to buy the full printed versions), he was horrified by the cost quoted by a reseller: £20,000 per year for the licence for the digital data. (OS says that it did not give a direct quote for that data). Trying to pay that would bankrupt the site.


instead, he turned to paper. “As long as you have purchased a ‘paper’ licence, you can scan maps and put them on the internet – as long as the webpage and the map doesn’t contain advertising,” he explains. He insists that the OS put this permission in writing – which it did, with a letter from a “senior service advisor”.

(We have seen a copy of the letter.)

This confirmed that “paper map extracts currently displayed on your website are covered by your Paper Map Copying Licence”. This, the letter says, is because “

  • the map extracts are being used as an information tool on your website to enhance your business
  • the mapping is not being sold and you are making no financial gain from the use
  • the map extract would need to be used in conjunction with the whole map sheet to give your extract context
  • there is no advertising on the same pages
  • “.

    Now, digital maps have far more utility than scans of paper ones – you can do all sorts of things with them that you simply can’t with a paper one, as Google and Streetmap and Multimap demonstrate. (You can’t dynamically calculate a route or a distance on a scanned map, unless you’ve done some very clever work locating corners.)

    But given that OS has to cover its costs, and that the paper licence only costs £50, it can’t be economical to issue the licence. It must cost more to administrate than is received. It would, surely, be cheaper and more efficient for OS to make paper map licences free.

    Which would be a start.

    Of course, as the article points out,

    this is not OS’s decision: ministers determine how it is funded. But with the trading fund model now being investigated, and OS under fire from a number of other government departments, the time is ripe for a radical revision of its funding regime.

    Time for ministers – perhaps with OS’s help, if it could say how much it costs to issue a paper licence – to have a think.

    (Afterthought: see, we do read your comments – and we do act.)

    We’re going to see the minister..

    Saturday, July 14th, 2007

    Perhaps the Free Our Data concept is getting some consideration in ministers’ minds. One of the first thing that Michael Wills did on being promoted to his new job at the Department of Justice was to ring us up and ask for a meeting.

    Eh? Ministers calling journalists? Quite a turnaround. Not even his private office, but the man himself.

    As a result we’re going to see him later this week, to make our case formally – and, perhaps, to see if there’s any formal response beyond that on the OFT’s Commercial Use of Public Information report, and perhaps the Cabinet Office response on the Power of Information.

    Among the examples we’re planning to use re government making things free providing a boost to commercial sectors:

    • GPS: the US government-funded Global Positioning System costs about $400m to run per year, but underpins a huge business reliant on its positioning data
    • museums: the Labour administration made admission to museums free. Why? To encourage more people to go to them. (The Conservatives propose to reverse it.. perhaps) It has led to more visits (though Hazel Blears points out that it’s not all been encouraging .. but a letter in June from senior museum directors says they like it – but that it needs funding. (Has anyone analysed the economic effects of 30m extra visits to museums – as in the train and bus journeys, the gifts bought, and so on?)
    • the examples of South Africa, Canada and New Zealand in making at least some of their data free

    Anyone else with the sort of arguments that would help a minister persuade the Treasury to unzip the purse strings and give the information economy an extra boost? Your suggestions – with sources, please – welcome.

    If councils move to Google Maps does that help or hinder Ordnance Survey?

    Thursday, May 31st, 2007

    Today’s Guardian Technology looks at how a number of councils, notably including the London Borough of Brent, and even some central government organisations, are moving to use Google Maps for their consumer-facing displays of map data.

    In Councils bypass Ordnance Survey for Google Maps, Heather Brooke looks at the shift, which councils are making because in the first instance, Google Maps is free and comparatively easy both to program and use:

    Traditional geographical information systems provide “complex data, complex systems”, said Dane Wright, IT service manager at Brent council in north London, at the annual conference of GIS in the Public Sector earlier this month. Google Maps, by contrast, provides “complex data, simple systems”.

    Wright told the conference: “What we are doing is moving to Google Maps as the primary interface for casual use by public users. This will leave the GIS system for more specialist users. The reason for doing this is to provide a better user experience – familiar interface, easy to use, integrated aerial imagery, attractive, no need for training or large manuals.”

    But, you say, OS is the source for pretty much all of Google Maps data – and where it isn’t then the company that is sources that from OS.

    That’s true – but it does mean that OS becomes vulnerable if Google decides that it would like to shift to someone else for its mapping data. And without knowing the precise details of the Google Maps licence with OS – does it pay per map displayed, per frame downloaded, or is it a lump sum? – one has to wonder what the effect will be.

    Meanwhile, even the government’s Directgov system for finding a school in a locality uses Google Maps (although a number of other Directgov systems don’t). Other examples of Google Maps (or indeed Yahoo! Maps or Live Local Maps) being used rather than OS for customer-facing products are welcome. Seen any?