Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens


Archive for February, 2010

Free Our Data response to the DCLG consultation on OS Free

Friday, February 26th, 2010

We’ve written up our response to the DCLG consultation, and emailed it over this afternoon. Make sure you respond too! Deadline is 17 March – a Wednesday, for no obvious reason.

Feel free to crib from this or build on it. Your comments welcome (though of course they’ll only be useful after the fact…)


Response to Ordnance Survey consultation from the Free Our Data campaign, 25 February 2010

Introduction
The Free Our Data campaign was co-founded in March 2006 by Charles Arthur and Michael Cross with the aim of persuading government that non-personal datasets created by government-owned agencies and companies and organisations should be made available for free reuse without licence restrictions.

The rationale for this approach is that citizens have already funded the existence and collection of these agencies through taxes paid over past years. (This includes historical data; Ordnance Survey, for example, has been a trading fund for some time but on its incarnation as a trading fund immediately used data previously collected at public expense.) Furthermore, the private and non-profit sector can imagine better ways of using data than government can because they have a direct interest in using it – but price and licensing are significant barriers to the development of those applications.

The campaign is apolitical. It is not aligned, associated with or funded by any political party or outside group; its (very small) costs are paid by the co-founders out of pocket.

We are delighted that government has chosen to accept the rationale behind the campaign’s logic with its plans to create the OS Free products. Our only caution is that it must ensure that the model used to fund it can be widely applied to other non-personal datasets within government. OS Free should not be a one-off, but instead should be the basis for a wider sharing of data.

One final, general point: the Free Our Data campaign believes that Ordnance Survey provides an excellent map-generating service and must remain a government-owned asset whose public task includes the continual mapping of the UK’s geography and built environment. Any moves to privatise any part of its operation would be retrograde and threaten both OS’s future usefulness and the UK’s economy. We would oppose such moves.

Question 1:What are your views or comments on the policy drivers for this consultation?
The need to reduce overt government spending, allied to the growth in personal computing power owned and controlled by the public at large, creates an entirely new opportunity to let citizens analyse, understand and benefit from the data that the government collects on their behalf. This is a two-way process.

Clearly the UK government is rapidly recognising the benefits of transparency – that actions are not just seen to be done, but that the reasoning for the actions can also be interrogated and understood. This is one key policy driver. (Hereafter PD1.)

There is a second policy driver (hereafter PD2): the need to reduce the public sector deficit in coming years. This is best done through a reduction in public spending and an increase in tax revenue from the private sector.

There is also an untapped private-sector entrepreneurial market whose entire existence depends on the successful implementation of this consultation and future ones like it. When government-collected data is treated as a limited asset which must be priced to create an artificial shortage, government constrains the private sector which generated the taxes used by the government to create the data. Clearly, that then constrains the tax base, because not all companies (extant or proposed) can afford to buy the data. Therefore total taxes are lowered by pricing data. This is inefficient, and constrains entrepreneurship based around the effective use of data.

Therefore making government-owned data like this free for reuse (including commercial reuse) will bring in larger tax revenues as long as HMRC is vigilant in collection of owed taxes from individuals and companies.

How the consultation will reduce public sector spending in the context of the Ordnance Survey’s financial model (as a trading fund) is less obvious, but still exists. The consultation iterates costs of making these mapping data free. However, it does not iterate the potential benefits through reduced costs to local councils, police forces and local health authorities, for example, of being able to provide map-linked data on public websites, without paying, at the Landranger and Explorer scale; this has been a consistent bugbear to local councils, to police forces and public health observatories which want to share their work with the public.

Making such data free also obviates the legal examination of any instance in which those bodies wish to share their work – a cost which is also unpriced in the consultation document. While these costs may not match the millions of pounds directly attributable in lost revenue from sales of Explorer and Landranger-scale data, they are significant in the cultural sense too – because they enable those bodies to operate in a more transparent manner as well, satisfying PD1 above.

Question 2: What are your views on how the market for geographic information has evolved recently and is likely to develop over the next 5-10 years?
The geographic information market has been completely transformed in the past 10 years by
-the opening of GPS (Global Positioning System) data to the world by the US military (an excellent example of treating “information as infrastructure”, in which the US government bears the cost of supplying, in effect, location data to non-US-taxpaying people in the UK and elsewhere); and
-the ability to create “crowdsourced” maps, such as OpenStreetMap (OSM), which are accessible via the internet without copyright restriction for consultation, addition or editing.

In the future the GI market will be further transformed by
-the growing number of smartphones with built-in GPS
-the falling cost of mapping large areas with great precision due to improving satellite photography systems
-mapping information, including road and route data, becoming a commodity, where only value-added forms can be effectively charged for.

Sales of satnav devices provide a clear indication that “knowing where you are” is a key piece of information for people: estimates suggest that between 4 million and 7.5m such devices have been sold in the past 10 years in the UK alone.

The commoditisation of route and road information, which has previously been supplied only by Ordnance Survey, will continue. If satnav makers decided that the OSM mapping was good enough, and that the pricing model it offers (of zero cost), they might choose to use its database (or even to improve its database while using it) and neglect the OS version. Under the present OS funding model, the only way for OS to recover its costs would be to raise costs to its existing clients, including the public sector – which would not fit PD2.

Therefore it is essential that OS does provide the OS Free data to encourage the growth of the UK geographical information sector, and develops its own high-quality mapping as part of the public task for which it exists.

Question 3: What are your views on the appropriate pricing model for Ordnance Survey products and services?
Given the name of the campaign, our obvious answer would be “all should be free”. But we recognise that there are pragmatic and political problems with this.

The question assumes a great deal about OS products and services, and its charging regimen. However as noted in the consultation OS has consistently declined to separate out the costs and revenues and profits of its “raw” and “value-added” products and services, which makes it difficult to take anything but a Gordian Knot approach to finding appropriate models.

The question would be better framed as “which products and services should OS produce, and what should it charge for, and how should the charging regime be set?”

The more logical approach is to ask what OS’s public task should be, what products and services flow from that, how far those should be self-financing (using, say, a trading fund method) and what other products and services are seen as a public good which should be funded out of general taxation.

OS’s public task is clearly to map the geography of the UK; arguably this also includes the built environment. MasterMap provides an appropriate starting point for the public task, comprising a detailed scale of the UK.

For the moment we find the proposed model – with MasterMap and non-OS Free products’ prices aligned for the public and private sectors – to be equable. However as costs of updating maps and built environment detail falls (due to pervasive GPS feedback systems such as smartphones and cheaper satellite imagery allied to automated updating of map databases) this may need review to see whether more detailed scale products closer to MasterMap level can also be offered free.

Question 4: What are your views and comments on public sector information regulation and policy, and the concepts of public task and good governance as they apply to Ordnance Survey?
PSI regulation and policy suffers from the problem that where public organisations decline to comply with it, neither method of enforcement is satisfactory.
-If OPSI or other organisations demand compliance using non-legal recourse (e.g. asking for “good practice”), the non-complying organisation can ignore it; or
-if OPSI or other organisations seek legal recourse for compliance, the exercise is extremely costly for all concerned and is concluded so slowly due to legal process that private organisations in particular are at risk of going out of business first. (The instance of Getmapping’s complaint against OS in the early 2000s is illustrative.)

It is absurd that OS has written its own definition of its public task – with or without the consent of its minister in DCLG. With the release of OS Free, it is time for the job of defining OS’s public task, which impinges on huge parts of British life and the economy, to be put in the hands of a body entirely outside OS.

Question 5: What are your views on and comments on the products under consideration for release for free re-use and the rationale for their inclusion?
It is essential that there should be both raster graphics and vector graphics. The former allow easy use on websites to create Google Maps-style interfaces (where the map can be “dragged” to a location). The latter allow dynamic scaling. Though no rationale has been offered for their inclusion, they seem to fit the “mid-scale” requirement.

The inclusion of Code-Point and Boundary-Line datasets, with licences that allow free reuse (including commercial reuse) is essential to the creation of useful, effective and profit-generation applications.

Question 6: How much do you think government should commit to funding the free product set? How might this be achieved?
This is a key question – and how the government chooses to implement this will demonstrate whether it is truly committed to the idea that “information is infrastructure” by creating a model of funding that will be applicable to other data-collecting trading funds and parts of citizen-funded government, or if it is simply choosing a short-term fix for the problem of the desperate need for free access to OS data.

It is easiest to start by indicating what the government should not do.

- It should not raise prices within government for the non-free OS datasets above those charged to commercial organisations outside government. This would create tensions under which government organisations would naturally seek third-party solutions to reduce their costs (because of PD2). That would undermine income for OS and jeopardise the quality of all its data. In extreme cases, price rises might deter local authorities and other public bodies from using high quality geographic information to deploy their resources more efficiently and end up costing the public purse more in the long term.

- It should not raise prices for commercial organisations above those charged to government for the same datasets. This too will tend to exacerbate any drift to third-party solutions for high-value datasets. (Although it should be expected that these will occur naturally due to new entrants in the market.)

Therefore government should commit exactly the “funding gap” that making the datasets mentioned free will cause – apart from the paper maps. OS will presumably continue to sell paper maps, and will be able to rely on its brand to benefit from their sales and consequent profits. Therefore Treasury should fund the “gap” in revenues out of general tax funding, rather than by levying greater charges for OS data from other public sector sources.

The government’s own argument that “information is infrastructure” should be applied here. Roads, for example, are physical infrastructure. Government sees their provision as a public good and commits to fund their building from general taxation. It does not charge higher road tax prices to government-owned vehicles to offset the fact that government has built the roads and provides free access to them. (Nor is road tax hypothecated towards road-building.)

In the same way, other public sector organisations that use OS data should not be charged over the amount that private sector groups would be, and their payments should not be hypothecated towards any “funding gap”. The amounts being discussed – ¬£19-¬£24m pa – are comparatively small when set against overall public spending.

The benefits, admittedly, are difficult to enumerate. It is possible that, as with GPS, the benefits will not be immediately visible, and may not appear in the same place as the investment. It would therefore be sensible for government to commission regular studies to evaluate the growth of business predicated on use of the OS Free products.

By adopting a “non-hypothecation” approach to funding OS Free, government will be greatly simplifying the process required for the subsequent release of other datasets from other government-owned bodies. The pressure to release OS Free arose because the trading fund model is too restrictive: it cannot prime the market.

To draw an analogy, the search engine Google could not be profitable if it were to use Microsoft’s Windows to power its multiple thousands of servers that store its index of the internet. It would have to pay a Windows licence on each of those servers, and for each additional one. The cost would outweigh its profits. Instead, Google uses the free Linux operating system for those servers. We are suggesting that using the OS trading fund model for products is akin to licensing Windows: it limits the size of the market for their use, and the speed with which companies can grow while using those products.

Question 7: What are your views on how free data from Ordnance Survey should be delivered?
The key to the datasets being useful will be (a) availability (b) reliability ( c) accessibility.
Availability: where are the datasets stored? If OS hosts the files, it will need to create an entirely new system to support hundreds or thousands of concurrent accesses. That is inefficient, and outside OS’s remit. It would be more sensible for the datasets to be uploaded to a cloud facility such as Amazon’s S3 storage or Google’s cloud facility where copies could be downloaded. This is a comparatively low-cost solution where OS would only have to pay for downloads, rather than setting up its own hosting service.

Furthermore, it is clear that the datasets will be subject to change over time. It would be inefficient to upload a complete set every day, for example. A more effective method would be to upload a “diff” file of differences from the previous full upload every so often (daily, weekly, monthly). This would reduce the total amount that would be needed to for an up-to-date download and simultaneously create new opportunities for applications showing what has changed on a map or dataset over time. A full dataset incorporating the diffs from the last full upload could be provided every, say, six months.

Reliability: so users can be confident that the files come from OS, they should be cryptographically signed.

Accessibility: the files should be made available in formats that are readable using open-source software: that will ensure that they will be usable by the widest possible range of users and applications.

Question 8: What are your views on the impact Ordnance Survey Free will have on the market?
Resellers of OS data will not be pleased. But this will force them to focus on value-added services rather than promulgating a system which perpetuates the extension of copyright limitations that are not sustainable in the age of the internet.

Some map providers have already cut their prices in response to the expectation of OS Free. As in Canada (in the example cited in the consultation) we should expect that mapmakers will take the opportunity to create specialised maps for different niche groups (climbers, walkers, and other outdoors pursuits are likely to be the first to take advantage of this).

The provision of CodePoint will galvanise a market that has been held back by the problems of creating fast, cheap and legal lookups for geocodes. Although organisations such as Yahoo offer them, using those leaves providers dependent on outside groups, when they would prefer to do their own lookup. CodePoint is an essential part of the package.

The provision of Boundary-Line will be highly important in the forthcoming election. It will also be important for online organisations which depend on mapping electoral constituencies.

Question 9: What are your comments on the proposal for a single National Address Register and suggestions for mechanisms to deliver it?
The absence of a working National Address Register (due mainly to intellectual property claims by publicly owned bodies) has created the absurd situation of the Office for National Statistics being forced to spend millions of pounds creating a one-off register for use in the 2011 census and then discarding it afterwards.

Government should retain the ONS census for future reuse and treat it as a resource with huge ability to create value for the economy.

Question 10: What are your views on the options outlined in this consultation?
Option 1 – allowing OS to continue with its planned “hybrid” strategy – is deeply unsatisfying. The strategy proposal has received no proper oversight; it has not been debated in Parliament; its financial assumptions are at best weak and at worst flawed; and the creation of an “attached” private company that would sell OS-branded goods is anticompetitive because it offers no transparency on pricing, while having sole advantage of the OS brand.

Option 2 – releasing large-scale data for free reuse – would cross a Rubicon. Although the Free Our Data campaign would support this, we are concerned that government and Treasury has not shown sufficient commitment to the idea of vote-funded data collection and parsing by OS, and that this strategy could endanger the long-term future of OS. Furthermore, it could undermine would-be commercial competitors, and would create substantial upheaval in the geographic information market. Change is good, but too much change can be unpalatable.

Option 3 – releasing “mid-scale” data as suggested, and considering a transition to further release – seems to offer a path towards the long-term future of OS while providing the opportunity to prove the benefit that would accrue to the private sector, and thus the Treasury through tax receipts, of freeing data. We find this the most pragmatic approach – but reiterate that the government’s aim should be to pursue a path where it releases data for the use of citizens without cost impairment.

Question 11: For local authorities: What will be the balance of impact of these proposals on your costs and revenues?
N/A.

Question 12: Will these proposals have any impact on race, gender or disability equalities?
We see no impact on those inequalities.

Charles Arthur & Michael Cross, 26 February 2010.