Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens

Archive for the 'Financial analysis' Category

Why is the government trying to corner the market for travel-direction sites?

Thursday, June 29th, 2006

In today’s Guardian we ask why it is that Transport Direct is using lots of data from local and central government (paid for by local and central taxes) to provide a service that has bugs – and is entering the market after a number of private ones.

In Time to tell this travel site where to go, Michael Cross investigates Transport Direct, which has had three million users this calendar year and uniquely offers point-to-point directions.

In an age when it is not seen as appropriate for the public sector to run power stations or railways, why is it running nationalised industries in what should be the most dynamic sector of all, the web-based knowledge economy? The question lies at the heart of our campaign, which argues that government’s role should be to collect and administer high-quality raw data, but make it freely available to everyone to create innovative services.

Transport isn’t the first area where the government has come late into such offerings:

Since its conception nearly a decade ago, “e-government” has been exempt from conventional political wisdom about competition, monopoly and state aid. The consequences are not only theoretical. In 2000, at the height of the dotcom boom, a London startup company called iMPower had the idea of launching a service to sell fishing licences on the web. It was supposed to usher in a new age of “intermediaries” providing electronic routes to public services. In theory, this was supported by government policy – but another government policy required the Environment Agency to launch its own fishing licence service on the web. The private-sector offering was unable to compete.

[Transport Direct’s chief executive Nick] Ilsley says that research by the department before Transport Direct’s launch showed the private sector wasn’t interested in providing a one-stop all-purpose site. However the site was launched in a market already populated by the private sector, albeit with less sophisticated offerings. Their operators argue that these are more in tune with public needs.

Our question: why not just make the feeds available for anyone to make use of, and build sites which could compete with each other, which would benefit taxpayers by generating revenues, rather than costing them for a function that sits on top of essential government?

How much does it cost to display a map online? The correct(ed) answer

Thursday, June 29th, 2006

After last week’s blog post in which we suggested it would cost millions to put an OS map of the UK online if you had a successful site (with, say, 2,000 users per day), we’ve had lots of people saying that’s wrong. And it is.

So today’s Guardian carries the story How much does it cost to display an OS map on a website?:

Much less than we estimated last week. In “Time to account for travel maps’ costs” (June 22), we wrote: “For a charity to put [those maps] on a webserver that might be used by hundreds of people (a typical server can handle 2,000) would cost millions of pounds annually.”

In fact, Ordnance Survey points out, the cost would be more like £18,000 per year – a hundredfold less than we suggested.

Of course we should have checked and checked again. Yup. We were following mySociety’s expectation that if it cost £1,000 for a single user internally per year, that that would scale up scarily once you displayed the data online.

But the confusion is understandable. OS’s page on copyright licensing for internet use (at does not specify that internet users en masse only count as one; hence mySociety’s expectation that internet use would lead to an explosion in costs.

Not so, said OS. For a single scale mapping of the country – say, at 1:50,000 scale – including Code Point for finding postcodes (so you can get a map of a postcode’s location), the annual licence for a website serving 20,000 map images per day, every day, would be £18,200 per year.

To provide the mapping service most web users are used to, one would have to license several scales: we have grown used to being able to zoom in on a point.

Streetmap offers seven mapping scales; Multimap offers 13, though not all appear to come from OS. Using multiple scales will, of course, ramp up costs very quickly – as will being popular. But even licensing seven scales will only take your annual costs to around £100,000 – not into the millions. Unless, of course, you are wildly successful.

So, that’s corrected, we hope. But even so:

Tom Steinberg, mySociety’s director, says: “The price for these maps, which cover only a small chunk of the country, is way above affordability by most small and medium-sized enterprises, a group that employs more than half the UK workforce. Furthermore, it excludes the entire caste of internet-based enthusiasts who’ve produced about 80% of all the innovative mapping work in the world over the past couple of years.”

Even at the new lower price, we probably won’t be putting those maps on soon..

How does Ordnance Survey justify its licensing costs when its accounts are disputed?

Thursday, June 22nd, 2006

In this week’s Guardian Technology supplement we have Time to account for travel maps’ costs, which looks at the costs of Ordnance Survey licences. If the organisation is right about the importance of its work (it claims to underpin £100 billion of economic activity), then – as Tom Steinberg of mySociety points out – even a slight error in overpricing the licence could mean a huge fall in tax revenues.

That is illustrated by the cost of a licence for “time travel maps” that mySociety created. As the article points out,

How much would it cost to put those maps on a webserver that anybody could access? MySociety asked Ordnance Survey’s licensing department. It calculated that displaying 16 “map tiles” with the relevant data would cost between £837.81 (for a 1:25,000 scale) or £1,032.71 (for a 1:250,000 scale).

Does that sound good? Here’s the sting in the OS’s small print: “All prices for one user, one-year licence and exclude VAT. Terms of data use are internal business use, display and promotion as long as there is no financial gain.” So for a charity to put that on a webserver that might be used by hundreds of people (a typical server can handle 2,000) would cost millions of pounds annually.

And here’s the other thing: the National Audit Office does not accept the OS’s accounts, and has not done since it became a trading fund in 1999, because OS treats the National Geographic Database (which originated with taxpayers’ money) as an intangible asset – but puts no value on it.

That in turn means that the £9.2m surplus the OS shows was almost four times above its target return, of 5.5% on capital employed (tangible and intangible assets). Yet put in the NAO’s estimate of the NGD’s value, and you still get a rate of return that’s about double the 5.5%.

What does that imply? To us, that OS licences are expensive – they’re generating too high a rate of return. Read and see what you think.

OECD meeting on public sector information: now online

Saturday, June 10th, 2006

The OECD recently held a meeting in Paris on public sector information and charging models. We were represented (ably, by Mike Cross) but haven’t yet digested it in its fullness.

However, you can find most of the presentations and topics at,2340,en_2649_34223_36860241_1_1_1_1,00.html. If you notice any particular elements that you think deserve to be highlighted, add them in the comments..

Free access to science speeds its use; would the same happen with government data?

Tuesday, May 16th, 2006

Though we haven’t made much reference to the issue here (previously), there is a huge debate going on in the scientific world about paid vs open access to scientific publications, and particularly papers. In a press release at Eurekalert, a European scientific resource for journalists and the public (OK, often they’re the same), a new release finds that

Free access to science speeds its use: “A longitudinal bibliometric analysis of citations to papers published in the PNAS between June 8, 2004 and December 20, 2004 reveals that the open-access articles were more immediately recognized and cited by peers.”

Now, in some ways this isn’t surprising. Articles that are immediately available to anyone with an internet connection anywhere in the world are more likely to be picked up and referenced than something which is either in print form only, or which can only be accessed through a paid-for website (or both). I know from my own experience that searching a site like PubMed can be very frustrating: you find lots of links to papers, but then run into the ground because the full text – which you want – is behind a paywall.

What relevance though does this have to the Free Our Data campaign? First, it’s certainly interesting that data which is freely available gets more widely used – even in a community as prolific about linking and referencing as science. That seems to us to bolster the argument in favour of making the data collected by government agencies available to citizens and businesses in the UK at no cost.

Secondly, we think the argument is more easily won in the Free Our Data case. Scientific publishers aren’t taxpayer-supported; the argument about where the revenue stream in a “free scientific publication” world is becomes complicated. Much simpler in the Freed Data world: the taxpayer supports the collection of the data; taxpayers then get to use the data, which creates more business, which generates more taxes, in a virtuous circle.

And we’re still waiting for the Treasury or someone to explain to use what the marginal social cost of raising funds actually means.

(Via EurekAlert! – Breaking News.)

How long does it take a government to do an economics study?

Thursday, May 4th, 2006

In today’s Guardian Technology we’ve looked at the strange case of the recommendation made in the Treasury’s Spending Review of 2000 that it should investigate the effects of lower pricing – perhaps even zero pricing – of government data.

And what did the authors find?

“The evidence is that certainly not all demands for government information are price sensitive,” the document, on the economics of government information, notes. “The government produces and sells value-added as well as raw data for which consumers are prepared to pay a premium while in other areas such as mapping and meteorology the government does much more than meet its needs.” That is, some of the data generated is surplus to the running of government; it’s simply there to generate profits, to offset the running costs of various departments.

…But the authors then admit it’s unclear how government-generated data should be priced. After discussing pricing models, they note the suggestion “that demand would grow rapidly in response to lower prices … and as basic information is repackaged in innovative ways. The issue needs further empirical work.” That is, an economist should see whether cheaper data boosts the economy.

And has anyone carried out that empirical study? Six years on, no, they haven’t. The Treasury says this is because there has been so much disruption in the public information economy, caused by the introduction of the Freedom of Information Act last year.

But we discovered that there is, meanwhile, a new OECD study on government data pricing (660KB PDF) – readers’ comments are welcome.

And we’re still trying to figure out what “marginal social cost” (see previous post) actually means in the real world.

What does the phrase “marginal social cost” mean?

Tuesday, May 2nd, 2006

Not being trained in economics, I’ve been wrestling with a phrase that I found in this part of the Treasury’s 2000 Spending Review.

Specifically, it talks about the idea of making data generated by government agencies free, which of course has to be done by replacing the revenues they lose (because they no longer sell that data commercially) with money raised from taxpayers.

The relevant paragraph (5.7) says in part:

If it did so however the Government would need to fund the resulting public expenditure from taxation. But taxation also normally mis-allocates society’s assets. Taxation absorbs resources and changes behaviour. The marginal social cost of raising public funds is generally agreed to be around 20-30% of the value of the extra tax receipts.

Emphasis my own. (How wonderful, by the way, to see the Treasury – particularly Gordon Brown’s Treasury – arguing against taxation. This is not a political point, but irony doesn’t come any richer.)

First, could someone explain what that phrase about the marginal social cost actually means? I suspect it means that there’s an overhead of 20-30% in running a public body compared to a private one. But I can’t find a clear example or definition.

Secondly, that figure seems to come from Ruggieri’s 1999 paper, which in its outline says it’s for “The marginal cost of public funds in closed and small open economies”. Does the UK count for economic purposes as a closed or small open economy? Your input very welcome.

Data pricing: more fuel for the fire

Thursday, April 6th, 2006

The following comments come from a reader who wishes to remain anonymous, but has given permission for them to appear:

I applaud your ‘Free our Data’ campaign – it is lifting the roof on a debate we have had rumbling on in our industry for years.

How about the example of Electoral Ward boundaries. My understanding is that the Boundary Commission captures this boundary data at tax payers expense, passes it on to OS (who may do some validation checking), who then sell it back to the public sector and anyone else bundled into a commercial product.

There really needs to be a set of core underlying geographic datasets (addresses, postcodes, wards etc) that are so key to efficient government that they are made free (or nominal cost) for all – this could be done under the OS NIMSA agreement although there’s likely to be major problems with data derived from Royal Mail as a commercial body. I would like to see core OS raster mapping down 1:250k, 1:50k, 1:25k and StreetView included in this ‘national interest data bundle’ to act as a set of base mapping widely available on a cost recovery or even free basis.

Contrast the Government position on funding OS with that of funding the Office for National Statistics (ONS) – very different. In a thoroughly forward-thinking move, ONS got Central Government funding to make Census 2001 data free for the first time at its point of use – see for more. This is in stark contrast to the position that it is ‘politically unpalatable’ to fund free data distribution. This move has freed up the market for increased use of the data and more suppliers to come into the area and create added value data products – the only problem is that if you want to do anything spatial with it you need geographic boundary data much of which is OS owned. So you are back to the problem of needing data from different parts of government and inconsistencies in data distribution models.

We can only hope recent changes at ONS to become a more neutral ‘arms length’ organisation will not mean a more commercial attitude is taken to data dissemination. There is no suggestion this will happen…yet.

How you get round legal issues of fair trading is not clear – right now there are few suppliers of large scale topographic map data for the UK with the exception to some extent of companies like Navtech and TeleAtlas although their focus has traditionally been on roads for navigational purposes. It would be very interesting to understand their take on this debate.

Let’s hope your campaign gets heard at the heart of government. It would all be much clearer if
  1. organisations that were wholly funded by government could only supply their data for free or on a cost recovery basis to anyone under relatively standard HMSO terms
  2. or

  3. organisations that were semi-funded by government and captured data using these public funds had to release it under the same HMSO terms – otherwise organisations would be free to sell it commercially
  4. If there are Trading Funds that want to go it alone (OS may be a candidate) then privatise them, keeping a chunk of shares under public ownership to reflect the intellectual property invested to date. At least this creates a level playing field. There are starting to be sufficient competitors in the market to give existing OS customers some realistic alternatives.

On a related point it would be great to get an independent accountant to assess the proportional value of the current OS intellectual property that results from long-term public investment compared with the proportion attributable to their own more recent self-funded investment. No doubt there is no simple calculation to do this but they do have methods. Of course the answer would be hotly debated but it might be a start to addressing the whole issue of “what’s ours” and “what’s theirs”.

Interesting – though I’ve also discovered in the past few days that NERC does make some of its data available for free. (I’ll do a separate post later, but it’s been pointed out that organisations like NERC are placed in a difficult position when they have to buy in commercial data to generate the datasets they use.)

What does everyone think? From the discussions here, one would imagine that people think the OS should veer more towards privatisation – which, to be honest, is not the aim of the campaign at all. (As we said in the original article launching the campaign, we think the UK should have a strong mapping agency: “if you view [the UK] instead as belonging to its taxpayers, and meriting rigorous mapping for their benefit, there are no “uneconomic areas” – only places that people haven’t started to use yet”.)

What’s the cost of sales and marketing in the government organisations?

Sunday, March 26th, 2006

Here’s a thought. We have our list of target organisations.

Now, could someone who has a few moments download their annual reports (really just the accounts) and look at their cost of sales and cost of marketing?

By our argument, the cost of sales is an unnecessary overhead which could be cut virtually to zero (to be replaced by cost of distribution of electronic data formats).

Cost of marketing would still exist: the Ordnance Survey, for example, would still want people to be aware of its different offerings, and its position as a prime supplier. (You can disagree – make the case in the comments.)

Does subtracting the cost of sales (and perhaps legal costs…) from the organisations’ costs substantially reduce their costs, and thus mean that it would be easier for them to “trade” without having to make sales? It’s a thought…

The money-go-round: how often does the government charge itself for its own data?

Friday, March 24th, 2006

An interesting email dropped into the inbox today..

One point which does not seem to have come across in the printed discussion is the question of other Government departments having to pay the OS to use OS data. This might seem bizarre – it is – but that is how things have come about. I was involved in negotiations with a number of Government departments and OS over a Service Level Agreement. Those of us who understood the huge potential benefits of GIS and linking datasets across departments were crying out for OS data but could not afford the fees OS wanted. Very many meetings were held between departments and OS, which resulted in reporting back to our own departments, then instigating discussions with sponsor departments to see if they would fund the use of data. The toing and froing was incredible. What it has cost the taxpayer in staff time far outweighs the income OS (and the Treasury) might receive. SLA’s now exist. Their terms are commercially sensitive. The cost is unbelievable. The whole process bogged down GIS development for 5 years and the enthusiasm of many hard working civil servants was drowned in the process.

(The sender requested anonymity).

This is an interesting point, because it’s clear from the Budget that Labour wants to push off any costs it can out of government – that is, not put any cash into organisations, and indeed to sweat the assets as hard as it can.

But if more than half of the OS’s £100-odd million in “revenues” is actually coming from other parts of government which aren’t “revenue-generating” (Defra? the Environment Agency – it uses OS maps to ilustrate its flood maps) or even private organisations which charge back to government (Capita uses postcodes for the congestion charge and for the TV Licence databases; would the cost of the contract to government be lower if the postcode or OS data were available free?) then something strange is going on.

That is, money is washing around in a sort of slush pile, and you never quite get to see where the music stops. If OS charges Capita for data, and Capita charges the government to run the TV licence, we arguably see no net benefit from charging for the data; it’s just an administrative exercise that makes work for the accounts people and the lawyers at OS.

Sure, Capita adds value to the data; that’s the idea of a private company. (No loans jokes please..) But this circular motion between government departments of money that only exists theoretically seems to me the weakest point of the government’s case for trading funds and charging for data – and remember, it’s that case which we’re trying to demolish.

Another day, another budget; but we need monetary arguments

Wednesday, March 22nd, 2006

I spent a long train journey last night reading an enormous email conversation about 2002’s withdrawal by the Ordnance Survey of the Panorama product (which provided comparatively cheap – though not free – data on height).

It was clear that the reason why the OS was withdrawing the product was simple: it competed with the higher-quality and more expensive products that it had generated more recently itself. Trouble was, people were still plumping for the cheap Panorama one.

So the arguments that were being made to the OS – that it ought to make the data available, that others would host the data for free if the OS would only release it to the public domain – clearly weren’t going to work.

What we have to recognise in this campaign is that it will not be won by appealing to the better nature of the OS or the Treasury. They don’t have one.

The only argument that wins is the economic one: that the Treasury (in particular) will receive more money through the data being free than by it being charged for. That means you have to demonstrate that there will be enough mapping companies, and that they’ll be big enough, that their tax revenues will exceed £105 million (the OS’s revenues, I think, in the past year).

OBviously here the Peter Weiss paper is the best illustration, but it’s only an illustration – at present. We need better illustrations tailored to the UK.

I’m considering the way forward at present. One tactic that seemed logical is to do an FOI request to each of the “target” organisations asking what data they charge for, and how much they charge, and how much revenue they get from them.

Other thoughts?