Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens

Archive for 2009

FOI request for OS to publish study delivered: the clock is ticking..

Monday, May 25th, 2009

Thanks to whatdotheyknow – the very public site where you can make Freedom of Information requests, assisted by MySociety (which previously brought you theyworkforyou and publicwhip, which track what Parliamentarians do in their work) – we have now filed our FOI request for the publication of the study that led OS to conclude that free data models don’t work.

The page is at and the request – which has to be answered by June 23 – is as follows:

On May 12 2009 Sir Rob Margetts, chairman of OS, said in a public
speech that “We did, with outside help, a review of equivalent
organisations around the world” in determining the effects of a
free-data model, mixed model or private model on OS’s future

I request the publication of all parts of the review that do not
contain commercial-in-confidence data, and the separate publication
of a full version of the review with commercial-in-confidence data

Very much looking forward to this.

Which foreign map organisations did OS visit last year for its study?

Saturday, May 23rd, 2009

An interesting question by Caroline Spelman: where did Ordnance Survey staff go in 2008?

Why might that be interesting? Because OS did a study of free data and other funding models last year as part of the preparation of its own internal study on which model would be best for the future. Would it be a free data model, would it be full private, would it be pretty much like the one it has? You may be able to guess. (Or read Sir Rob Margett’s speech as I recorded it. Or watch the OS version of it – they chopped out the detail of his speech and his assertions about cost to the government.)

(Apparently there was a “brisk” question-and-answer session. My question about derived data didn’t make it into the video.)

Now here’s the document answering Ms Spelman’s question. I’ve highlighted a few answers in bold. Can you guess why?

And does anyone know what funding models the mapping agencies of Finland, Canada, New Zealand and Estonia (Estonia??) operate?

Plus – have I missed any? There is of course one country missing from that list which should have been visited but wasn’t. Interesting to know why…

Countries visited by Ordnance Survey staff in 2008 (original:






Representing Ordnance Survey and United Kingdom at an international conference on geographic information



Participation in a technical working group on the INSPIRE Directive


Participation in technical working group on the INSPIRE Directive


UK Delegate to International Cartographic Association Conference


Technical participation in Semantic Technology Conference


Technical participation in European Semantic Technology Conference 2008



Keynote speech by Director General, representing UK at Middle East Survey Technology Conference



Participation in Geographic Information awareness event in the European Parliament


Technical participation in Mercator print equipment User Group Conference, sharing best practice


Technical participation on behalf of UK in European Parliament meeting on Global Monitoring for Environment and Security (GMES)


Technical meeting with European Commission


Technical meeting with European Commission (Directorate on General Information Society and Media)


Representing Ordnance Survey at European Centre for Public Affairs conference


Technical meeting with team drafting the implementing rule for European Commission on the INSPIRE Directive


Technical meeting with the UK Permanent Representation Team


Technical meeting on the Shared Environmental Information System (SEIS)


Participation in EuroGeographics management meeting


Technical participation in European Reference Framework Conference (EUREF) 2008


Participation in European Public Sector Information (ePSI) conference



Technical meeting with National Resources Canada


Technical meeting of Geographic Information Web Networks (Geoweb)



UK Delegate to the International Society for Photogrammetric and Remote Sensing (ISPRS) conference.



UK Representation (including presentation by Director General) at EuroGeographics General Assembly 2007



Technical participation in meeting on the INSPIRE Directive


Technical participation in meeting of the International Standards Organisation Technical Committee (ISO TC211)



Client visit Korec – supplier of surveying equipment


Attendance at Society of Local Authority Chief Executives (Solace) conference


Transit departure airport for return from Dublin conference



Technical meeting with National Land Board ‚ exchanging best practice and fact finding.



Technical participation in meeting of the Business Interoperability Group of EuroGeographics


Technical meeting with Finnish Land Survey‚ exchanging best practice and fact finding.


Visit to supplier to discuss paper stocks for printing paper maps.



Representing Ordnance Survey at a EuroGeographics business meeting


Technical participation in meeting on the INSPIRE Directive


Representing Ordnance Survey at a EuroGeographics business meeting


UK Delegate to the International Cartographic Association Workshop on Generalisation and Multiple Representation, and the Spatial Data Handling (SDH) Conference.


Business and sales attendance at the International property trade Conference.



Technical meeting with CityGML ‚ regarding the production of 3D City Models


Technical participation in a EuroGeographics Business Interoperability Group meeting


Sales attendance at the Frankfurt Book Fair


Technical attendance at CeBit Conference – Information and Digital Technology


Technical participation in the Spatial Cognition 2008 conference, including participation in a workshop designing maps for orientation.


Technical and Business participation in an Open Geospatial Consortium meeting.


Technical participation at a 3D Special Interest Group meeting at University of Bonn, supporting Ordnance Survey’s contribution to the creation of the CityGML data format.


Technical participation in the Spatial Cognition 08 conference


Technical visit to the DRUPA printing equipment exhibition ‚ for fact finding.


Pan-European conference on Geographical Information related matters.


Technical participation in a meeting of the pan-European EuroRoadS Project Management Board.



Technical attendance at international technology analysis “Canalys” Conference



Technical visit to Supplier


Technical visit to Supplier



Technical and business participation in a meeting of Open Geospatial Consortium Technical Committee


Technical participation in a working group on TRI-Partite multimedia Object Description (TRIPOD) at Cantinetta

Technical participation in International conference on knowledge engineering – EKAW



Technical presentation to the European / Japanese Conference on Information Modelling and knowledge bases



Technical participation in a meeting on TRIPOD



UK Delegate and presentation by Director General on “Perspectives on the challenges facing the geospatial industry; a view from a National Mapping Agency”.



Technical meeting with the European Commission (Directorate on General Information Society and Media on public sector information)


Technical participation in a meeting of the European Commission on Geographic Information Systems


Technical and sales participation in the “Apply Serious Gaming Conference” related to the use of high specification geographic information by the gaming industry.


Technical participation in a meeting on TRIPOD


Technical participation in an Ontology Web Language Experiences and Directions workshop



Representing Ordnance Survey at EuroGeographics business meeting



Technical participation in Urban Land Institute housing affordability seminar


Representing the Ordnance Survey and the UK the Director General gave a keynote speech at GIN Congress


Technical participation in Workflow and Production Management Technology Conference 08


Airport for technical meeting on the development of the INSPIRE Directive at Hague


Technical participation at the 3d GeoInfo 2007 conference. Furthering expertise and knowledge of 3D data collection and systems.


Technical meeting on the INSPIRE Directive


Technical participation in Data Quality Workshop at ITC, Enschede


Research meeting with AND (Automotive Navigation Design) on use of Ordnance Survey data in navigation solutions.

New Zealand


Representing the UK and Ordnance Survey at an international conference on conceptual modelling and to visit LINZ (Land Information New Zealand), the national mapping and cadastre agency of New Zealand.



Technical Participation in meeting of Euro Spatial Data Research (SDR)



Representing the UK and Ordnance Survey, the Director General gave a keynote speech on geographic information and national decision making at Elblag Conference



Representing Ordnance Survey at EuroGeographics business meeting



Representing the UK and Ordnance Survey, the Director General gave a keynote speech at EuroGeographics meeting


Representing Ordnance Survey at business Meeting



Technical participation at conference on the INSPIRE Directive



Technical and business participation in Barcelona Euro SDR meeting


Technical participation and sharing of best practice in Photogrammetric Digital Camera user forum


Technical participation at 3GSM Congress


Technical participation at meeting of ORCHESTRA – European collaborative research project on spatial data infrastructure


A four month secondment to the Instituto Geogr√°fico Nacional (IGN) Spain to gain a better understanding of how Spatial Data Infrastructures can be implemented, both organisationally and technically. This was in return for a secondment of an IGN expert to Ordnance Survey in 2007.



Technical participation in pan European technical project work associated with INSPIRE legislation


Technical participation in Mercator print equipment User Group Conference, sharing best practice


Technical participation at conference on the INSPIRE directive


UK delegate at Federation International des Geometries (FIG) Conference


Technical participation at Tobii Conference



Technical and business visit to Leica Geosystems


Two visits to ETH Zurich University, one regarding the quality assurance of 3D buildings and attendance of a conference at ETH presenting Ordnance Survey work and best practice on the capture of 3D buildings.



Technical participation and representation of Ordnance Survey at the Usability Professionals Association Conference


Technical participation at Vital Vision Conference


Technical participation and sharing of best practice at Association for Manufacturing Excellence Conference


Technical participation at GI Science conference


Technical participation in international conference in geographic information


Sharing of best practice at Conference on SAP systems in Human Resource models

Salt Lake City

Technical participation in conference and workshops at GIScience 2008 ‚ sharing expertise of geographic information science and geographic information systems.

San Diego

Representing Ordnance Survey at BAE System user conference

San Diego

Technical participation at the ESRI conference on GIS and mapping software

San Francisco

Technical and business participation at the Oracle user conference

San Francisco

Technical and business participation at the Association for the Advancement of Artificial Intelligence (AAAI) Workshop

San Francisco

Technical and business participation at Where 2.0 Conference.


Technical participation in discussions at ION GNSS 2008


Technical meeting with Microsoft on the use of Ordnance Survey data.

St. Louis

Representing the UK and Ordnance Survey at Open Geospatial Consortium meeting

St. Louis

Representing the UK and Ordnance Survey, the Director General held a technical meeting with the National Geospatial Intelligence Agency


Technical participation in OWLED / Washington Workshop

OS chairman’s speech: internal study shows “free” OS would cost government 500m-1bn pounds – but won’t publish

Thursday, May 14th, 2009

The following is the text – as captured in shorthand contemporaneously – of a speech by Sir Rob Margetts, chairman of Ordnance Survey on Tuesday May 12. It is not complete but does capture the major themes and quotations.

The context is that Sir Rob was explaining to an invited audience, including many existing customers of OS, how the new “hybrid” strategy had been determined as the best one for its future development. He took some pains to emphasise that the “free data” model had not been rejected out of hand; but that instead a special study had been commissioned to investigate it.

This is my shorthand notes of what was said. My own comments are at the end.

There were major issues affecting the sustainability of OS as it goes through its proposed strategy.

We examined the complete range of options very impartially and objectively. That includes the free data, utility model where you would make data available to anybody [for free]. We examined the fully commercial model.. and alternatives within that range.

Our study of the utility [free data] model was done because some hold that that is a good strategy, and some of us weren’t indifferent to it. Some [of the study team] going in thought it could be interesting.

The study was fully costed for the government, calculating the costs of change to the residual value.

We came to conclusion that the cost to government in the first five years would be between £500m and £1 billion. That wasn’t the only reason that we discarded it. We did, with outside help, a review of equivalent organisations around the world.

We wanted sustainability and high [data] quality and came to the conclusion that at nearly every organisation that had gone to free data model, the quality had declined and that users and customers were increasingly dissatisfied with the product.

And the attractiveness to staff and recruitment and retention had also reduced. We found no evidence that this model actually worked elsewhere.

Those that work had a user-pays model. We tried to understand and explain why. Think that comes to the responsiveness to needs of the organisation. [ie: the responsiveness of the organisation to needs.]

If customers are required to pay then they specify needs very clearly and give feedback on whether they have got value [for money].

Customer stimulation is a vital part of any organisation because it’s sustainable.

And of course [there’s] recruiting and retaining quality staff.. they want to work for a qulity organisation and respond to real customer needs.

That’s why we didn’t pursue [the free data model] but can affirm that we looked at it in detail.

We also looked at a fully commercial model but weren’t satisfied it would fulfil the fundamental strategy [for OS].

We believe use [of geographical data] has expanded dramatically and changed.. but that potential is still considerably underexploited.

Our No. 1 aim is to improve capacity of OS to assist the exploitation of geographic information and be one of fundamental enablers of that [exploitation] in the UK for social and individual benefit.

With the proviso that by doing that we have to keep a sustainable organisation that not only covers its costs but also has enough left over… about £20m per annum.. to invest in the products that the market needs for customers, whether private individuals or business enterprises.

Commentary: Well, we’re fascinated to learn that OS found that there’s absolutely nobody out there who is making a free data model work. We have already emailed the South African mapping organisation, about which we wrote in 2007, to find out whether they were contacted by OS, and if so what they told them.

We will also pursue Freedom Of Information enquiries to find out which organisations OS spoke to and what their responses were. Since these are all free data models, there can’t be any commercial confidentiality for the foreign organisations, can there?

The “£500m – £1bn” range is extremely wide, and we’d like to see the detailed working. I asked the minister with responsibility for OS, Iain Wright, who was there, if he would order OS to release its full study. He said that if there weren’t any commercial-in-confidence implications… I wonder if we’ll see it? Again, we’ll ready some FOI requests.

There were questions at the end, and one interesting one came from Bob Barr, who pointed out that there is always the possibility of “pay to change” – that when you have a database of 460m features with (to give the statistics that Vanessa Lawrence, OS’s chief executive, read) 5,000 changes daily, why not charge those who are changing it? (We’ve looked at that model before, though I would like to see some more recent Land Registry figures.)

Here’s the question as I recorded it.

Robert Barr: “this hybrid financing.. it seems to be today that payment will be at the point of use. Usually [in other online systems] there’s a model where you pay to change the database. Doesn’t it make sense for data to be paid for where you change it?”

Peter ter Harr of OS: “This is a model we have been looking at. There are advantages and disadvantages. It’s not always the user who pays [in the current model]. There are many OS products which are free at the point of use. It’s the information provider who puts it online who pays. We have been looking at the model in various other countries. It works well in cases where it’s part of the statutory process.”

And that’s it? We really, really need to see that OS internal study, as it contradicts pretty much every study that’s been published. It’s going to be fascinating tracking it down.

One other thing: the cost to the government isn’t quite the same as the benefit to the economy, nor the eventual benefit to the government through taxation. It was the latter (actually, both) that the Cambridge study looked at. We are perfectly happy to generate tweaked versions of the “free data” model that could keep OS charging for some products (such as MasterMap) while freeing other data sets. Now that would be a truly hybrid model.

If anyone has had sight of that OS study, or any part of it, do please drop me an email at Or upload it to Wikileaks and let us know. We think it’s so important it ought to be out there, not locked away in an OS cupboard.

Government hits free data decision into the long grass

Thursday, April 23rd, 2009

We have the inside track on what’s going to happen at Ordnance Survey – which will be formally announced this morning.

Today’s Guardian says, in Government ducks free data decision:

The government has kicked into touch a decision on the future of its largest state-owned digital information business. The Communities and Local Government department will today announce that the Ordnance Survey must make more of its data available to re-users – while apparently grooming part of the agency for future privatisation.

The new business strategy, published the day after the budget, follows a review by the Treasury’s Shareholder Executive. The headline finding is that “a model where a user pays a licence fee for OS data continues to be the most effective way of balancing the need to increase the availability of geographic information to the wider UK economy and society while maintaining the quality of OS data”.

But in a concession long called for by the Free Our Data campaign and others, boundaries information will be available for free as part of an extended “OS OpenSpace” service. Also available will be some OS products “from 1:10,000 scale through to 1:1 million scale”. The MasterMap database will remain proprietary.

Here’s what is going to happen: a new commercial arm of the OS (but without ownership, and having to pay just like its commercial rivals for OS data); and more emphasis on OpenSpace (but not so much that it would actually compete with any commercial versions).

Our opinion: a complete and utter shot into the long grass. Ducked the issue. Shied away at the last fence. Until we see clear evidence otherwise, it’s an indication that even though the government has made encouraging noises about seeing the value of making data free, and even though it has received a report that it commissioned which showed that making data free would bring huge economic benefits, it can’t quite make itself believe it. Better to bail out banks with tens of billions that you might never get back than spend a few millions stimulating commercial enterprises and encouraging entrepreneurship by giving people access to essential, business-valuable data.

We’ll have more analysis and reaction when all the documents are available.

In the Guardian: Ordnance Survey’s future awaits budget; Peoples’ Map launches

Thursday, April 2nd, 2009

In the Guardian, Michael Cross notes that Ordnance Survey’s future is quite possibly going to be determined by the Budget.

In Ordnance Survey’s future to be mapped out on budget day, he notes that

The future of British government’s largest digital data business, the mapping agency Ordnance Survey, looks set to enter the mainstream political agenda for the first time in a decade. On budget day, 22 April, the Treasury is expected to release the broad findings of the Shareholder Executive’s review of the “trading fund” model of funding agencies such as Ordnance Survey and the Land Registry.

This is of course the review that Adam Afriyie of the Conservatives demanded the government should publish. A little tardy, one might argue. Why not publish the review when it’s ready, rather than amidst the Budget, when there are a million other things that need far closer examination?

The review is likely to shine a spotlight on anomalies created when government bodies function as businesses in the digital economy. It will present ministers with three choices – outright privatisation, a move to supplying data at marginal cost (“free data”) or splitting the organisation up.


Whatever the findings of the Shareholder Executive’s review, Ordnance Survey is likely to use its ability to generate cash returns as an argument for continuing as a trading fund. However, the agency’s apparent profitability will encourage calls for outright privatisation.

We also note the launch of the Peoples’ Map, which

allows users to create their own maps by drawing over aerial photographs. Getmapping, the company behind the venture, described the product as “the democratisation of the mapping process”.

It’s an intriguing idea, though personally I can’t quite see where its utility comes in; OpenStreetMap already offers an extremely detailed map for all sorts of parts of the world; the Guardian is using OSM to some extent, and some local councils are using it to map their footpaths.

Still, for Peoples’ Map,

A big selling point is simple licensing terms. The company says that maps generated on the system will be free for private non-commercial use apart from a delivery charge of £25. Commercial users will have “fair perpetual licensing arrangements … and entirely free of third party copyright” – a reference to the byzantine intellectual property regime surrounding many products containing OS-derived data.

Ah, yes, OS-derived IP rules. It’s a fascinating subject to which we will return.

Islington: you want a map? You’ll have to pay

Tuesday, March 24th, 2009

An intriguing move by Islington Council, which now demands that if you want to download something that has a map in, you pay. At least if that something is part of a planning proposal.

From its Planning Policy page:

Islington’s planning policies are set out in our Unitary Development Plan (UDP) – which was agreed in June 2002. The UDP provides the basis for all the council’s planning decisions. It contains broad strategic policies in part one of the plan, and more detailed policies in part two.

There is also a proposals map which shows areas where specific policies and proposals apply. You can see the text of the UDP on this site, but if you require the proposals map you will have to purchase a copy. [Emphasis added – CA.]

So why has Islington introduced this? (We may have been a little remiss in noticing this – the page says it was last updated on 31 October 2008.)

Could it be because of that famous Ordnance Survey warning of last year, made around that time? Is the council prohibited from providing any sort of map, or perhaps charged each time someone downloads a map? And if the latter is the case, is it making a profit, breaking even, or loss on the transaction?

(The page for the proposals map is confusing too. “click on ‘Interactive Maps’ on the top right hand side of the screen. A new window will open up.” it instructs. However, on my browser – a Firefox clone on a Mac – I don’t get any such “interactive maps” link. Is this a PC-only thing, or has that page just not been updated to keep tabs with the pricier new world? Its last update is the same as the parent page, so it surely can’t be…)

Treasury tightens the screws on OS: will job losses follow?

Monday, March 16th, 2009

Always useful to monitor the written and spoken questions in Parliament: if you use (and who wouldn’t?) you can set up an alert each time a phrase is used in Parliament (either house), or an MP appears, and get taken to the relevant page.

The latest interesting fact to emerge is that Ordnance Survey is expected, for the financial year just ended, to up its return on capital employed (ROCE) from the usual 5.5% to 6%:

Adam Afriyie (Shadow Minister, Innovation, Universities and Skills; Windsor, Conservative): To ask the Secretary of State for Communities and Local Government what financial return on ordinary activities was expected from Ordnance Survey in the year ending July 2008.

Iain Wright (Parliamentary Under-Secretary, Department for Communities and Local Government; Hartlepool, Labour): The financial target for the year ended 31 March 2008 was derived from the three-year target for 2004-05 to 2007-08 set down in the Ordnance Survey Framework Document 2004. This target was to achieve no less than 5.5 per cent. per annum return on capital employed (“ROCE”), averaged over a three-year period, with the return defined as surplus on ordinary activities before interest and dividends.

The target figure for ROCE for the year ended 31 March 2009 was increased to a return of not less than 6 per cent.

(Emphasis added)

That’s interesting, and if we get a moment it might be useful to see what that sort of increase in ROCE means for the OS’s profits and especially costs: we have heard that Vanessa Lawrence, OS’s chief, has warned staff that there might be redundancies in the coming year.

It might seem perverse to you – it does to me – to increase the ROCE demanded from an organisation that relies on third-party sales of its products as a recession bites, which would tend to mean that (a) those third parties are going to have less money to spend (b) some of those third parties might go out of business. “Capital employed” tends to be a fixed number (staff aren’t capital, they’re operating costs), and is hard to change.

In the Guardian: one-off census address database cost rises 20% to 12m pounds

Monday, March 9th, 2009

In The Guardian, we’ve followed up on the question put by the Tories’ social affairs minister (if I recall correctly) Eric Pickles, who asked about the price of the one-off census address database for 2011.

It turns out it has risen from £10 million to £12m in the space of just a few short months. Hasn’t anyone told them there’s a deflationary recession on?

As a reminder: the need for this database has grown from the fact that the Ordnance Survey, Royal Mail and local authorities can’t agree on how to build an address database that the Office for National Statistics can agree will be definitive for carrying out the 2011 census.

Well, fair enough, you might say. The ONS’s requirements differ subtly from OS, RM and local authorities. It’s possible that their interests and plans won’t be entirely congruent.

What’s mind-boggling, and completely idiotic, is that ONS is going to build this database for itself (must already be in the process, since it doesn’t have long to do it) and then is going to destroy it. Because nobody could agree to let such a thing exist independently.

It’s an egregrious waste of money – first the building, which is madness, given that the data already exists; and then the destruction. Occam’s Razor of datasets: don’t let entities multiply needlessly. And the first law of not wasting things: don’t destroy hard-won datasets needlessly.

In the Guardian: what happens to the Postcode Address File in a Royal Mail split ownership?

Monday, March 9th, 2009

With Lord Vold… Mandelson looking to persuade a private partner, likely TNT, to take a minority shareholding in the Royal Mail, the interesting question arises of what happens to the ownership of the intellectual property of the Postcode Address File (PAF).

After all, if you buy into a company, you’d probably want to see more efficient use of its assets. (That’s part of the plan in the shareholding selloff.) Would that mean that TNT or similar would start trying to “sweat the assets” of the PAF?

In What does the Royal Mail sell-off mean for postcodes data we investigate this briefly. The problem is that nobody – including the Turner Report into the future of the Royal Mail – seems to have considered this. Neither PAF nor intellectual property nor postcodes are mentioned at all in the Turner report.

PAF is profitable –

in August 2007 the postal regulator Postcomm revealed that PAF operations made a profit of £1.58m on revenues of £18.36m, all but £4m from resellers.

However marketing organisations, which use PAF, don’t like the idea of those assets being sweated.

“The reason for getting the private sector involved [in the Royal Mail] is to improve efficiency,” said Robert Keitch, director of media channel development at the Direct Marketing Association. Raising PAF prices would make it harder to check addresses and increase the need for manual checks by postal staff, he suggested.

Our opinion?

The Free Our Data campaign has consistently suggested that the PAF – linked to map data – should be made available for free, without copyright restrictions, due to its growing importance for location-based services. The comparatively small cost of running it, especially without the costs of administering its sales and checking for violations of licences, could perhaps be borne through a levy on address or name changes, or simply through the tax revenues that could be gained from new companies set up to take advantage of the datasets. However, it is unknown whether Mandelson will recommend that.

We await developments.

Tories demand publication of Trading Funds review, back free data principles

Thursday, February 19th, 2009

photo: Paul Downey

The Conservative Party is demanding that the government publishes the Trading Funds Review, while also giving its backing to the concept of free data – although it hasn’t quite gone as far as backing the entire principles of the Free Our Data campaign.

At a debate last week hosted by the Policy Exchange (a broadly right-wing thinktank), Charles Arthur – that’s me – Steven Feldman, Ed Parsons, Shane O’Neill and – crucially – the shadow skills minister Adam Afriyie debated the idea of free data and what it could mean for government.

Perhaps one problem was that there wasn’t anyone to represent the other side of the debate. I had thought that Steven Feldman would oppose the idea – but he didn’t, and doesn’t.

But the meat of the evening was in Afriyie’s speech. Now, the Free Our Data campaign isn’t party political; the more sides back it, the more likely we are to get change.

So, to Afriyie’s speech, which he presented pretty much as it’s on the site:

The campaign to ‘free our data’ is an important one – all the more so at a time when our economy is in deep recession.

And further:

So my vision is for a more open, innovative and better connected society: a society where access to communication technology creates more powerful citizens and a less controlling state; a society where the free-flow of public information energizes entrepreneurs and social innovators.

Data will be the fuel of this new economy. And as the repository of the country’s public data, the Government already has a vital presence in the field.

A key point:

First, I turn to non-trading fund data. This is the deep well of raw data which lies unpublished, unavailable and untapped in dusty government vaults.

This includes raw data for crime maps and school league tables. It could include hospital performance statistics and patterns of carbon emissions. It might include the coordinates of every mobile phone mast, held by Ofcom, or the location of accident black spots, held by the Department for Transport.

One little point: the mobile phone mast data doesn’t actually belong to the government. It really belongs to the mobile phone companies, but I think that the government has an interest in holding it. This was one of the topics that OnOneMap brought to the fore.

Afriyie continued:

There is any number of data sets below the radar, and I’m sure members of the audience will have their own ideas. Much of it can be mashed-up and re-used, adding value for the economy and society.

And now to the crunch stuff.

Second, I turn to trading fund data. This is information which government agencies sell to generate income. The Ordnance Survey, the Met Office, the Hydrographic Office and the Land Registry are good examples.

Now let me be clear: we see nothing intrinsically wrong with asking users to pay for a service. Nor do we oppose attempts to replicate efficient, business forces within the public sector.

But we are instinctively cautious of government monopolies. And that puts us in a position to ask whether there could be a better way to generate more jobs, more wealth and more enterprise for the nation by doing things differently.

Take Ordnance Survey as an example. While there is a case for the government having responsibility for mapping the nation’s territory, with growing opportunities in the digital economy it’s not so obvious that the government should also run map shops. Conservatives do not usually take the view that retail businesses are best run by governments.

That’s an interesting point. It’s indicative of the importance and breadth of what the trading funds do that they are still owned and run by the government.

Afriyie notes that there has been plenty of studies about the benefits of free data models, including the Cambridge study and more recently the Trading Funds Assessment, carried out last summer and autumn – just as markets and banks properly went into meltdown.

But so far there have been at least five government reports. And some – like the long delayed Trading Fund Assessment – have been announced with great fanfare only to fade with a whimper.

Indeed we are now told, bizarrely, that the Trading Fund Assessment will not be published and will be treated as private advice to Ministers. There’s open government for you! But we do welcome to the Trading Fund Assessment. The situation should certainly be scrutinised. But the review should not be an excuse for kicking the issue into the long grass.

So I’m calling on the Government to publish the review, and let us have the debate.And let’s be clear: it should not be an excuse for half-hearted measures.

It is of course important to read the whole speech (this is only a few extracts from it) to get the full idea of what the Conservatives are – and aren’t – saying here. It’s a nudge in the direction of a free data model, and Afriyie’s reaction on the night seemed to indicate that he thinks it’s a good idea. But it’s a long way between the idea and the implementation. We’re not done yet.

(Paul Downey has a good summing up of the event too. It’s his picture at the top.)

February 10: come to the public debate on free data, government agencies and copyright

Friday, January 30th, 2009

On Tuesday 10 February at 5pm, the think tank Policy Exchange will be hosting a debate entitled Free Our Data? Government Agencies and Copyright, in Westminster.

Speakers will include the Technology Guardian editor, Charles Arthur, a co-founder of the Free Our Data campaign; Adam Afriyie, shadow minister for innovation; and Ed Parsons, geospatial technologist at Google and former chief technology officer at Ordnance Survey. We’re hopeful that there will be at least one other speaker, from the public sector.

If you would like to attend, email (You’ll discover the precise venue then!)

Wouldn’t it be wonderful to have this data for the UK?

Sunday, January 18th, 2009

The Guardian has pulled together a collection of datasets drawn from the US:

Simon Rogers gathered this information and shared the raw data via Google Spreadsheets for anyone to use. This means that people can grab the data in whatever format is most desirable including text, .csv, .xls, and .pdf.

Since access is open on each spreadsheet, it also means that developers can write client applications that interact directly with the data. Developers can access the same source data as either XML or JSON.

Fantastic stuff. Now, wouldn’t it be marvellous if we could find the same datasets for the UK and know that we could all share it for people to build on?

If you do know of any copyright- or charge-free (or both) sources for these for the UK, then please leave us a note in the comments.

In the Guardian: the mystery of the vanishing addresses

Friday, January 9th, 2009

Thursday’s Guardian Technology looks at the mysterious ways in which addresses within some postcodes are simply vanishing from the Postcode Address File (PAF) – that enormously useful index of places in the UK which can receive mail. A new Highland clearance? Well, sort of.

In January 2008, the picturesque west Highland village of Applecross contained 32 buildings with postal addresses. A year on, it has only 24. This is not the result of some new Highland clearance, but an absurd consequence of UK government bodies treating data collected in the course of their work as a commercial asset rather than a national resource to be shared.

It’s not that the houses are going away; they’re very much still there. And they’re still owned by people. But Royal Mail, in attempting to maximise the value of the PAF, is removing them because that makes PAF more valuable to direct marketing companies – even while it reduces its utility to local authorities, which initially gave Royal Mail the details of the addresses, because they need to know about habitable locations in order to do things like emergency planning and other local services (dustbin/recycling runs, anyone?).

Royal Mail says it has a policy of removing addresses from the database when houses are unoccupied. “If the postie can no longer reach the delivery point, or if a house is obviously completely unoccupied, the postie informs us and the address is removed from the PAF. If it later becomes occupied, it would be put back on.”

And then…

Turf wars between Royal Mail, local authorities and Ordnance Survey over the ownership of postal addresses have a long history, imperilling everything from emergency services to the national census. Local authorities are particularly bitter about the current state of affairs because they have the statutory job of creating addresses in the first place. As one council specialist put it: “Local authorities create addresses, Royal Mail adds the postcode – then this data is sold back to us by Royal Mail and Ordnance Survey.”

Any other examples of this that anyone has come across?

A quick roundup to start the new year

Thursday, January 1st, 2009

Hope you’ve all come through the new year without suffering too many leap-last-year problems. I thought it would be interesting to round up a few things that I’ve seen but not really had enough brainpower to turn into anything more than notes.

First, Public Data Sets stored on Amazon Web Services. (Via Richard Allan.) An interesting idea: got public datasets? Well, why not get them stored somewhere really cheap where people can access them but you only pay per download. It’s the ultimate outsourcing, and you also get to see how many people are downloading it without the capital costs of the servers.

Public Data Sets on AWS provides a centralized repository of public data sets that can be seamlessly integrated into AWS cloud-based applications. AWS is hosting the public data sets at no charge for the community, and like all AWS services, users pay only for the compute and storage they use for their own applications. An initial list of data sets is already available, and more will be added soon.

It’s already got the Human Genome and the US Census data. The idea of hosting UK public datasets on AWS was floated in the Cambridge Economics report released with the Budget back in March. Any takers?

Second, Municipalities open their GIS systems to citizens (thanks, Gerry Gavigan, who points out that “As well as innovation and the other usual unexpected benefits, it points to the existence, alas without quantification, of financial benefits.”) The article explains:

For instance, the online burning permit sales service of the Minnesota Department of Natural Resources (DNR) allows citizens to declare precisely where they would like to burn woody debris. High precision is essential in deciding whether a permit is obtainable, as well as when and under what conditions: if there is a high fire risk in the area and day for which a user asks for a permit, the software must refuse it. The Web site, however, makes it easy to enter the location with the greatest possible resolution: users first type an address into a form to get an approximate location on the map, then zoom at will and finally click on the exact spot for which they are applying for a permit.

And, more pertinently:

The success of initiatives like OpenStreetMap or the availability of Yahoo! and Google Maps APIs may make you think that people may create services like these and many more all by themselves, without getting any bureaucrat involved. However, in order to benefit the most from digital maps and other spatial data, citizens need such data to be officially inserted in, and completely integrated with, the maps and databases public administrations use to plan roads, zoning, and everything else.

Citizens may use Web sites like those mentioned here to request services as different as bus stops, trekking permits, or new post offices. Other uses may include signalling construction abuses, damages to public property, or illegal dumpsters. We may draw our preferred public bus routes on a map in our City Council official Web site.

Of course, to make all this work in practice, public administrations should also clarify the data ownership situation. Who owns data directly and freely provided from citizens? What license should apply to those data or any derived ones? This, however, is a separate issue, not really related to open source software.

And finally, some interesting questions being asked in Parliament by John Howell (of the Tories) about Ordnance Survey income from local authorities, and on use of OS vector data for commercial use (and, previously, about discussions between OS and Google over mapping licences; Ed Parsons, formerly OS and now Google, says the minister’s answer is wrong); while Mike Gapes, a Labour MP, about London mapping payments to OS and payments to OS for use of its data by various government authorities.

We’d be interested in any comments on what Gapes and Howell are trying to unearth here… and of course your comments on anything else. And happy new year!