Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens


Archive for the 'General' Category

A chance to tell OPSI what we want

Friday, January 4th, 2008

Our friends at the National Archives are making an energetic start to the new year, with a series of events to sound out what re-users of public sector information want from the custodians of our digital crown jewels. The first is a “barcamp” to be held in London a week on Saturday. (A barcamp sounds like one of those establishments on Old Compton Street, but apparently is a new kind of participatory “unconference”.)

Here’s the official announcement.

“The Office of Public Sector Information, part of The National Archives, is holding a conference to ask re-users of public sector information to shape the future of public sector information re-use.  The event is open to anyone interested in public sector re-use and will take place on the 12 January 2008, at the Spey & Ness Rooms, City Inn.

“The aim of the web channel is to present public sector information with a commercial value in a user friendly way that will encourage its re-use and simplify its uptake, by improving interaction between departments and end-users.  The format of the final channel will have been shaped by the user community contributions through the web channel forum (at http://www.opsi.gov.uk/forums/forums/index.asp) and this BarCamp ‘unconference’.

“The event follows the decision to launch a web-based channel as recommended by the Power of Information report that will “improve the Government’s responsiveness to demands for public sector information”. If you would like to attend the event you can register and shape the agenda by visiting http://barcamp.org/BarCampPOIR8.

“The ‘unconference’ will take place at Spey & Ness Rooms, City Inn, 30 John Islip Street, Westminster, London.” 

Michael Cross will be there for the Free our Data campaign. Say hello!

OBE for Robert Barr, free data campaigner

Monday, December 31st, 2007

We don’t want to be too bold in claiming influence in high places, but we’re delighted to record that one of the inspirational figures behind this campaign has picked up a gong in the New Year’s honours list.

Congratulations go to Dr Robert Barr, of Manchester Geomatics and Manchester University, on his OBE for services to geography. Among his many other activities, Bob is a passionate campaigner for a saner approach to the management of public sector information, in particular address databases. We hope he won’t mind being counted as a (sometimes critical) friend.

Perhaps he’ll put in a word for free data when he drops in at SW1A 1AA…

Is OS’s OpenSpace all it’s cracked up to be?

Wednesday, December 26th, 2007

Merry Christmas to our dedicated readers; and apologies for not updating more often – I did blog the launch of Ordnance Survey’s OpenSpace on the Guardian Technology blog, and thought it had been quite well covered. But we could have done it here too..

Anyway, people have had time to (as they say in the US) kick the tyres of OpenSpace. What do they think?

Ogleearth is not impressed because of the restrictions on volume. The OpenSpace FAQ says:

1.4 How much data am I allowed to use?OS OpenSpace allows your API key to access up to 30 000 tiles of data and up to 1 000 place name look-ups per day for free.

5.4 My site gets a lot of traffic, what can I do?Congratulations :! Contact us to talk about the different possibilities we may be able to offer you. If you are interested in commercialising your application, take a look at point 6.2 of the FAQs.

To OgleEarth, which is a well-read blog, that means that

So let’s say I add an embedded OpenSpace map to a blog post on Ogle Earth that shows 9 tiles. That should last me about the first 12 hours of every day. Sorry late-rising Californians, my quota is up! Should I add two such maps within 15 posts of each other, so that the front page of the blog displays a total of 18 squares, then Europe is out of luck too. Translation from OSese: Feel free to have a website that uses our maps, as long as it is obscure and unpopular.

Mm. And it adds:

this is so stupid and tone-deaf to the realities of Web 2.0 that I’m practically sputtering into my Malay gin & tonic. Pages and websites are artificial constructs in Web 2.0, as we’ve now all moved on to services aggregated from wherever they may originate. Do I really have to set up an ad-free www.NotOgleEarth.com and link to maps served from there as popups or iframes every time I want to use OpenSpace? Is OS not aware that advertising such as Google ads is ubiquitous on the blogs and hobby sites that the OpenSpace API is ostensibly for? For the overwhelming majority of bloggers — certainly those for whom 30,000 tiles per day is plenty — the ads at most help defray hosting costs, or else they are injected by web hosters to pay for free hosting services.

That’s not really a great vote of confidence. What do other people say? A Technorati search brings up a long list of results – with people who are happy that OS has done this.

The OpenStreetMap crowd (that’s a compliment – eyes, bugs, shallow) analysed the licence and found that if you generate something new, using information from the OS, which is “a severable improvement” (as in, you can separate it from the map, and it’s better), then it’s your data. A small win…

TechCrunch UK wasn’t impressed, because it doesn’t go far enough for them:

However, it is not great news for startups. The OS has “almost? come to its senses because there remains the issue that startups will not be able to create commercial businesses out of this data from the word go. Even though these are businesses from which the government could potentially extract tax revenues, again.

That’s straight out of the Free Our Data playbook, though we have to say we’ve not been in touch with TCUK. (Maybe we should.)

Blacksworld notes that the OS cartography has far more detail, which is a very pertinent point – and the one which is behind us pushing for this data to be more easily usable.

As its author notes:

The license is a license, so some people and going to love it, some will hate it and most will just get on with hacking. One thing that the guys from the OS were emphasising yesterday, was that they really want people to consult with them. That’s why we were there yesterday – so that the people who made OpenSpace could see what we thought. I think the desire of the OpenSpace team to listen to people’s feedback and act on it is a genuine one, so maybe we could all try some constructive criticism before trashing it. But hey, this is the internet.

It’s a good point. The OpenSpace API could be better; it could be unlimited; but then again, the OS has to pay for its data costs somehow. While it’s a trading fund, it can’t just give away the use of its data centres. We’ll have to wait for a more enlightened regime, perhaps.

Does this sound familiar? Virtual London removed from Second Life – at Ordnance Survey request

Tuesday, November 20th, 2007

The Virtual London project has once again fallen afoul of Ordnance Survey, which this time has spread its domain of dominion into virtual worlds that have no physical existence.

So, the virtual London that the UCL team had built in Second Life has had to be un-built. The team explains:

Our Virtual London model in Second Life has been removed from the collaborative environment at the request of the Ordnance Survey.

The research is currently ‘pending license clearance’ as the Ordnance Survey are ‘uncomfortable’ with the use of the data.

Details on the work currently unavailable are in the post below, we are reserving comment at request on this one, but i guess you know our views…

Three Dimensional Collaborative Geographic Information Systems (3DC/GIS) are in their infancy, Google Earth opened up the concept of three dimensions to the mainstream but issues with data copyright, the inability to effectively tag data to buildings and the asynchronous nature of the platform have limited developments.

Second Life however provides a synchronous platform with the ability to tie information, actions and rules to objects opening the possibility of a true multi-user geographical information system. It has been notoriously difficult to import 3D data into the Second Life but at CASA we have managed to import our Virtual London model of 3 million plus buildings into a scrolling map. The map is built from prims that ‘res’ our of a central point to build accurate models based on Ordnance Survey MasterMap with height data supplied by InfoTerra.

We’re very interested by the concept that you can infringe copyright with data in a virtual world. Then again, maps are the original virtual worlds, aren’t they?

APPSI chief comes out swinging against lack of government information/data policy

Monday, November 12th, 2007

Richard Susskind, head of the government’s Advisory Panel on Public Sector Information, has been rather critical of government’s lack of strategy over PSI. We’ll have a longer interview with him in the paper on Thursday, but meanwhile here’s some extracts from a position paper (or lack of position paper?) that he has just completed. You can find it on the APPSI’s “Papers for Ministers” slot, or linked below here on the site.

“This paper is written primarily for Ministers who have a direct interest in or responsibility for the re-use of public sector information. It considers why and how the Government should formulate an explicit strategy for the re-use of PSI”

, it begins.

and continues:

“There has been growing recognition in recent years that this information (for example, geographical, meteorological and statistical information) constitutes a resource of great potential value; that public information is an asset, an intellectual asset, that should not be seen as usable for one purpose only. Instead, it is argued, this information can and should be made available for re-use (a recycling of sorts).”

Under current obstacles, he identifies

  • disagreement over the value of reuse (“APPSI strongly and confidently challenges the OFT”): Susskind says that “APPSI argues that the impact of the re-use of PSI is very significantly understated in that document and is better understood not by calculating the revenue generated by licensing PSI but by identifying the extent to which businesses and institutions rely on PSI.”
  • unclear government priorities: does it want to cash in on PSI to reduce the net public expenditure, or strengthen UK industry’s competitiveness?
  • There has been no single focal point for PSI, at Ministerial level, to ensure coherence across the public sector.” (This was one of the remarks made – with some amazement – by Derek Wyatt at the RSA/Free Our Data debate in summer 2006.)
  • The rapid growth of the Internet and emerging online developments may render current policy out of date.” (Already has, we would argue: the internet tends to make available any data and information resource that is reliable and free and elevate them above paid-for resources.)
  • There is significant disagreement over charging policy.” That is: “Some support trading funds and their sale of PSI in the manner of private sector businesses. Others argue for making PSI available at no cost or marginal cost. And still others make a case between these two poles.” Could he possibly mean us?
  • most damning, “There has been, until recently, little interest in the re-use of PSI amongst most Ministers and senior officials.” Although, he adds approvingly, “Recent ministerial interest suggests this may be a problem of the past.” Certainly if the response from Michael Wills is anything to go by.

The question of privatising some of the trading funds (first in line might be UK Hydrographic Office) is moot, he says, “in the absence of clearer underpinning policy” [on PSI]. That is, if you don’t know quite how valuable your PSI is – and what UKHO generates is definitely public-, not private-sector information – then you can’t make an informed decision about whether the taxpayer (who is the present shareholder in UKHO) will benefit from privatising it. If you reduce the shareholding to those who buy the shares in the privatised company, does that benefit the people who used to own it (us) by more than we lose?

His short-term list of jobs for the government is that within a year, the Government should:

  1. undertake or commission a robust analysis of the actual and potential impact

of PSI re-use on the UK economy and society, supported by plausible

valuations;

  • prioritise clearly which classes of re-user should be the prime beneficiaries
  • and identify what tangible benefits it expects to accrue;

  • nominate a senior Minister to champion the systematic and coherent re-use of PSI across the public sector;
  • assess the impact on PSI re-use of existing and emerging Internet-based technologies, especially Web 2.0;
  • revisit charging policy in relation to the licensing of PSI with particular
  • reference to trading funds 6; [already being done; report due November 22 – CA]

  • increase awareness of the impact and value of PSI within and beyond the
  • public sector.

    As I said, we’ll have a longer interview with Richard Susskind in Thursday’s issue. Meanwhile, you can read the document from here on the site.

    Interview with Michael Wills: full transcript

    Friday, July 27th, 2007

    The following is a transcript of the meeting with Michael Wills, minister for information, with representatives of the Free Our Data campaign. It’s been checked against the recording. There is a lot in here about government intentions if you read between the lines, particularly towards the end. Please do try to read it in full – and your comments are welcome.

    Interview with Michael Wills, minister for information, 19 July 2007.

    Present: Charles Arthur (editor, Guardian Technology); Michael Wills, minister for information; Michael Cross, Guardian Technology; a representative of the Office of Public Sector Information (OPSI); press officers and office staff for Michael Wills.

    Charles Arthur: Thanks, first of all, for setting up this meeting. We spoke to Baroness Ashton previously before the response to the [Office of Fair Trading’s] CUPI [Commercial Use of Public Information] report and the Power Of Information report; the responses were interesting. We really wanted to just talk to you about what our campaign aims to do and to some extent sound you out about what you think about where the use of government data goes, to put it into context with the freedom of information, we see the data side as being part of the same spectrum, but we’re very focussed on the impersonal side, which we see as having tremendous value when used commercially. The best example I think of a government project which provides data completely free which then generates economic benefits is GPS, which is a project that costs about $500m per year but according to evidence that’s been put forward in the House [of Commons] underpins the US economy; in the UK people use it for sat-nav that generates business, that generates taxes – so in effect we get a free ride from the US government. There are similar benefits in making data available for free for businesses to use because the businesses can then grow; if you give the data away free it might seem you’re forgoing all sorts of tax revenues but there’s a multiplier effect, that you then get a bigger economy and tax, and there’s a benefit….. and that’s our pitch.

    Michael Wills: Well, first of all thanks very much for coming in. Look, it’s a compelling pitch, that’s the first thing, and personally I’m very excited by this area, I asked to do this as part of my portfolio, when you become a minister … technically my title is the constitutional affairs minister. I wanted this – the Prime Minister appointed me to come and do the Constitutional affairs job, which is a big green paper that’s going to take a lot of my time, here and energy, but when I looked at the division of portfolios with the secretary of state I saw this area and I wanted to do it, I was previously responsible in one of my ministerial incarnations for FOI. The whole issue of data is I think tremendously exciting for all the reasons that you’ve said, it’s part of the infrastructure now of our society and our economy and it’s going to become more so with what’s happening with data mashing, the extraordinary intellectual creative energy that’s being unleashed is something that as a government we have to respond to, and the power of information you know is a very exciting document, something that I think is very much where government wants to be. Now, not in every detail, we have to work policy through very carefully, we have to be sure that what we do is going to work, is going to be effective, is going to be cost-effective and we will come back to some of those issues in a moment. But as a broad approach we are very sympathetic to that. I met Tom Steinberg last week to talk about his ideas of how we can move forward in a whole range of areas, it’s very exciting, and we have to respond, we want to respond but we have to–. So those are the key headline points, so overall we are very sympathetic.

    There are two issues really that we’ve got to address and they’re linked, I think– you stop me if you want to interject, but I think there are two issues we’ve got to address with this. One is the basic issue that all governments have to address in all areas of public policy: who pays, essentially. And as part of that we’ve got to be clear about what is the most economically efficient way of doing this. Now, in terms of the economic case, the case is on the face of it compelling, but compelling cases don’t always stand up to intense scrutiny. We can’t rush after our instincts on this; we have to have the job done carefully, properly, thoroughly, which we are doing. We’re committed to producing a study by the end of the year. We should be able to do that.

    Michael Cross: Do we know the terms of reference? Do we know who’s carrying it out?

    MW: I don’t know quite yet but we will deliver it by the end of the year, some things are commercially confidential, you understand but but but .. we have committed to carrying it out by the end of the – we will deliver this and in the end, look, we’ll be judged by this, this will be a public report, it will be open to everybody to asses, we will take views on it, we will be very clear about the openness of the consultation on the report and then we will make a judgement on where we go with it. There’s nothing, you know, secret about this, we have to, we have to make this case, whatever we decide to do, because whatever happens there will be someone will be paying the cost and whoever it is we have to justify that. Whether it’s the general taxpayer – because it is seen to be something that’s appropriate by the general taxpayer – partly because of all the arguments you’ve just made, but then we have to make the case for the general taxpayer; if the trading fund model persists we have to make the case to you and your campaigners and all the rest of it. But we will be judged by this report and we will engage with it properly. I don’t know what it will say, but it will be on the table and then we will move forward, and we’re not hanging around; as I say we should be ready by the end of the year. I know that can seem quite a long time in this world, but government–

    MC: Presumably you need to tender for the consultants to come in and carry it out–

    MW: They’re all being put out to public procurement–

    MC: The terms of reference–

    MW: In government terms this is moving pretty quickly.

    MC: What I was pressing for is the terms of reference, which by implication would have to be ready pretty imminently

    MW: We’re on course, OK. We’re really on course, we’re not hanging around on this because look, this world is changing so fast, and in a sense every year that goes by and we haven’t got this finalised one way or the other — it’s very, it’s a potentially very important decision, this is part of the infrastructure, and you’ve got to get this right. I think we’ve moved very positively already; we want to go on with that. And, now, just as an example of that, I just want to, OPSI are now, we’ve had discussions already about what we can do, they’re going to be setting up a web-based channel to gather and assess requests for public sector information–

    MC: –that was one of the recommendations of the Power Of Information report–

    MW: — and we would like you to become involved in shaping how we develop that. So I’d like to know if you would after this talk to [OPSI’s representative] and take that forward and – it’s not instead of, you understand, all the others things, but it’s a first step, that’s something we can do now. And how quickly [OPSI] do you think we can get this up?

    OPSI: We’re committed to July 2008, and from our point of view which is in response, we’re looking to do it as quickly as we can–

    MW: Could we do it quicker?

    OPSI: With a fair wind (MW: with a fair wind) – and it depends a little bit on what the community says that they want in terms of how exactly this thing works, because it’s really important to us to get it right.

    MC: A lot of work has been done in this area by the Demographics User Group and obviously Locus so we could come up with a user requirement, very quickly

    MW: That would be comprehensive as far as you were concerned, you reckon?

    MC: It would cover commercial use; we’d probably want something coming from the academic users, the association of geographical information–

    CA: And probably also the non-commercial users, individuals rather like Tom Steinberg started out as being, who have great ideas and expertise.

    MC: We could come up with a users’ manifesto pretty quickly I should think.

    MW: In a sense the quicker, the sooner we will move this.

    OPSI: As fast as we can.

    MW: But wanting to get everything done properly. We want to move forward on this.

    MC: You were saying there were two big issues – one was the affordability.

    MW: The other is the economic case; not the affordability, but who pays, which goes with the issue of the trading funds and all that; the world has changed dramatically since the 1970s [when trading funds were set up] and we have to reassess, that’s absolutely clear, but we can’t prejudge the study.

    MC: As you know there are a number of quite urgent competition concerns coming up – what can you do to make sure that these are addressed?

    MW: Well, they will be addressed. I have to address them; it’s not an option, we have to address them. I prefer to address them looking at this contextually and not be driven purely by reaction, which – we will react, we have to react, but it would be much better if we could have a comprehensive, holistic policy which also is flexible. This world is going to change again, so to some extent we have to, when we look forward, not get locked in. I don’t see why we should, I’m not saying we will, but — what I think we have to do, we have to be aware that this world is going to go on changing, probably faster than we think. And so we have to have an immediate response; we want to create a holistic approach next year. Realistically we’ll have the study ready; we will react very quickly to that, depending exactly what it says, and we can’t prejudge it, but depending exactly what it says, there’s no reason why we shouldn’t move very quickly–

    MC: One problem here is–

    MW: –our officials as you know are very good, very active, they want to get this right, so we should be able to move very quickly next year – [OPSI] you’ll tell me if I’m over-committing you all but we can move very fast once we get the study in place.

    OPSI: We have the [POI] recommendations, we have quite a tight timetable for reporting December 2007 to PA16? and then again in December 2008, and the machine is working bringing together the officials that are involved in that…

    MW: There’s no reason why that should slip at all is there?

    OPSI: None at all.

    MC: One complication though in this obviously is that we have the INSPIRE, the implementation of INSPIRE, has its own transpositions into UK legislation, its own timetable and that’s a complication you always get, isn’t it?

    MW: Well, it’s one of the things we’ll have to take into account, I’m afraid all public policy sector … competing, not competing, but differing inputs and we’ll have to manage that. I’m sorry, again, I’m not being evasive but I can’t be precise until we’ve seen what the study actually says.

    MC: Yess.. but we already have concerns, most startling I think in Defra and the MOD submission to the CLG [Communities and Local Government] select committee looking at the future of Ordnance Survey; in the context of INSPIRE it’s Defra’s submission which is actually quite forthright about the frustrations of the trading fund model in trying to implement INSPIRE, and these things that seem to require more urgent action than is coming from a holistic policy coming out next year.

    MW: More urgent than next year? Well..

    CA: To some extent, I think what the select committee submission gives Defra the chance to do is to make its feeling about Ordnance Survey public, and it gives it a forum in which to do it and gives it a reason why it should be doing it, but the sort of evidence that we’ve found in the 18 months or so since we’ve been doing the campaign is that there are all sorts of little bits within the public sector who are very annoyed — Ordnance Survey gets picked on a lot because it’s the biggest PSIH, the biggest trading fund, the biggest revenue–

    MW: And very successful–

    CA: And very successful, yes absolutely, within those terms of reference, but there are enormous frustrations at what people find they can’t do, There’s the North Eastern Public Health Observatory which wanted to do a map-based, in effect a mashup, of morbidity, mortality, various sorts of things, and found that they couldn’t, and that frustrated them and still frustrates them, it’s still up there as a big hole in what they want to do, and it comes to things like flood map data; floods, obviously, a billion and a half pounds gone down the drain in the past couple of months. The Environment Agency is in effect constrained from how it can licence its flood map data, partly because of the way it uses Ordnance Survey data; again and again we come across these sorts of frustrations with the fact that the trading fund model puts this sort of requirement on the trading fund itself.

    MW: Well, what you’re describing very graphically is the opportunities that are there. That’s why we’ve got to move, that’s why the government are so positive about this agenda, you’re just proving again the point that there are these fantastic opportunities. Look at it slightly differently, from the point of view of government we accept the opportunity, the case for moving on this, we’ve got to, and we are, already. Government inevitably, and rightly, moves quite slowly and cautiously, we’re dealing on behalf of all the people we serve, and you can’t be rash about this. In this particular area government is moving, for government, very fast precisely because we do all see the opportunities that you just described. They are there. And what’s so exciting about all this is that you know that even though you can’t specify all these things, once you let this creative energy go, people will come up with extraordinary things out of this. None of us around this table could predict it will be, or exactly when, but we know that in the next 5 to 10 years extraordinary things will come out if we can unleash this creative energy in the right way, so the point is absolutely taken, there’s no question about that, and we will move as quickly as we can. But, having said that, from looking at the need to move, develop, find the right way to unleash all this energy, to actually what the model should be, is – you can’t do it in one step. We have got to just take a little bit of time, and I mean a little bit of time, we’re not pushing it into the long grass, they are very tight timescales, the study will be available at the end of this year… this is rapid.

    MC: Do you concede that both CUPI’s report and our anecdotal evidence suggests there is something seriously wrong with the trading fund model as it now works in the information market?

    MW: Well I prefer not to put it like that, what I would prefer to say is that there are huge opportunities out there and we have to be certain that we’ve got the right way of realising them.

    MC: Of course, yes–

    MW: That’s the way you’ve got to look forward–

    MC: But you’d concede there’s prima facie that there are a lot of problems with the current model.

    CA: In effect a market failure.

    MW: We admit, well, we think that there’s prima facie evidence that we need to look at this again, that’s what we would say, I think. Of course we do. There’s no question – the trading fund model was established 30-odd years ago. It would be extremely foolish of us not to look at it again, which we are.

    CA: Will the – I couldn’t quite work out whether the TOR have been drawn up or are being drawn up, will the examination of the trading funds look at other countries’ experiences or will it be limited solely to the UK trying to determine whether it’s efficient or inefficient in the UK?

    MW: Other countries’ experiences are always interesting – you’ve got a piece today on New Zealand, which is interesting – but they’re all slightly different, and there are all sorts of reasons why this will have to focus on this country. One of the reasons why we’ve just — it’s important to remember that we have very high quality public sector data in this country and the trading fund model though it was set up a long time ago has clearly been doing something right in that way. The reason that this is so exciting now is partly because we actually have such riches here produced under the trading fund model. Now, that is not an argument for no change, please don’t misinterpret that, but it is saying that we just have to be careful that as and when we change, that we don’t lose some things that are precious and — I could give you, but I won’t, a defence of the trading fund model because I think we’re now looking at all this again, but we have to recognise that however, whenever it was formulated it has produced something very important. Now, the world is completely different now and we can’t just assume that because something worked for the last 30 years it’s going to work for the next 30 years, it won’t, it will not, and therefore we have to change, and that’s why I’m slightly careful. Now the economic case that is produced is just part of the case; we will be guided by it, obviously, that’s why it’s been commissioned, but it won’t be definitive; I just say that, we will listen very carefully to the responses that we get; it will be one study, it will be an important study, but we will be listening to the comments on it and the discussion; we have to be sure that we’re getting this right, we’ll only do that by consulting and engaging and assessing the responses fully, openly.

    MC: The other question apart from the TOR is picking who’s going to carry it out; you can imaging in a small world of experts.. it’s a sort of snakepit of jealousies…

    MW: It’s not just in this sector (laughs). As I say, we have to get this right; that is our criterion. I can assure you I have no set view on what the outcome should be. Honestly, I want to see the study, I want to hear the responses to the study, then want to decide. But the responses are going to be as important as the study itself, if that’s any reassurance to you. I understand why you’re pushing on this, and you’re absolutely right to do so; there are various things I can say now and various things I can’t quite say right at the moment.

    MC: Who will actually appoint the – what shall we call them, the reviewers.

    MW: Government.

    MC: Government. Will – you have the resources of APPSI–

    MW: I’m the minister responsible for this; there are a number of government departments which have a keen interest in this, and we have to work with them. We have to be very clear that we’re setting up a model that is going to be sustainable. The worst thing that we could do right now would be to set up a model that we would have to tear up and do again in two years, because that doesn’t provide the stability. If we’re looking at economic benefit, the one crucial thing is that the model has to be sustainable. Otherwise you won’t get the investment, you won’t get the energy; people are, investors particularly, always wondering if this [business] is going to be sustainable. You won’t get the venture capital – that is not indispensable – but as we know from other areas in this world, if you get creative risk-taking venture capital involved, then that is a crucial factor in unleashing creativity. So we have to have a sustainable model, and that means we have to take Whitehall as a whole with us and [it] has to be signed up to generally. Although I’m responsible.

    CA: So does that mean that Treasury has an input in terms of how–

    MW: I think you’ll find Treasury has an input into everything in Whitehall.

    MC: To move on, what’s your feelings about where the public sector should be involved in the knowledge economy? One of the things we – perhaps it’s a personal hobbyhorse – is a website called Transport Direct–

    MW: Oh, yes.

    MC: –which is a classic government effort, technically it’s probably brilliant, though there are all sorts of things about it routing wrong – it’s the classic all-singing all-dancing, expensive, probably available in Welsh, government work project but it squeezed a lot of private sector people doing this sort of stuff out of the market and has a chilling effect on the market for travel information ever since. Do you think government is right to be doing this sort of thing?

    MW: I’m not going to comment on Transport Direct because I don’t know about it. What I’m about to say should not be said to be about Transport Direct, and I can’t make a judgement. Look – government has to be involved, and is involved, because for all sorts of reasons, in this area particularly it generates huge amounts of data; it has to – and therefore you can’t remove government out of it. But I think we have to be extremely careful about the role that government plays in markets, and there is always a risk that the state can stifle innovation and creativity; it can be innovative and creative in its own way and can produce things, but — you probably don’t know, but I used to run my own business in television, so I have operated in that sort of intensively competitive marketplace for longer than I have as a politician, and not surprisingly I believe in its virtues. I operated in a sector when there were lots and lots of really quite small, highly creative, highly professional companies, all fighting each other for really quite a relatively small market, intensively competitive. And perhaps – you’ll forgive my personal bias – but I thought it worked quite well you get a lot creativity, television in this country is constantly renewing itself, but it is a highly creative industry which attracts lots and lots of very creative people, which has gone on doing that for generation after generation and it works, and the explosion of activity that took place in the 1980s and 1990s, after liberalisation of the market, I thought was very productive, So – and I don’t think that, but — having said that in this area, that I do know well, I think the BBC remains having a fundamental role; it has to, and it goes back to what I was saying about the infrastructure, there is always going to be a role for government in providing some sort of infrastructure.

    MC: There is a role for government–

    MW: You can’t remove government’s role in this particular area because we are a player, and cannot help but be so; there is also a positive role in providing the infrastructure, which we’re doing, which all this is about, providing the proper infrastructure so all this can take place, But beyond that, I think you have to go with very great care as government. By and large the presumption must always be is that you let people go and follow their own instincts, let them make some money; if it works – great. That’s the way you’re going to get the creativity and the energy and the unexpected consequences, you can’t do these things top down; everything tells us that if you let people just get on with it something wonderful is going to happen. If you start saying what it is that’s wonderful that should happen, you won’t get something wonderful.

    MC: So given that would you accept that over the past, say, five to seven years for all sorts of good reasons government has perhaps moved too far up the value chain in PSI?

    MW: Well, I could tell you – no, I won’t tell you the story. I’ve got a lot of very young officials around the table who have long lives ahead of them and I don’t want to stop them by giving them a heart attack right now. I’m not going to tell. But, I think we have to be very careful as government; any official body has to be very careful about entering into a marketplace. We have to be exactly clear, scrupulously clear about the reasons why we’re doing it and we should never just go into it because we think it’s a good idea or might be interesting; and by being clear about it – what we can contribute to it, that we’re not going to stifle creativity and innovation coming up from the grassroots.

    CA: Our argument isn’t and has never been that OS should be sold off or should be hived off in any way – our argument is that Ordnance Survey is a terrific organisation that does great work, and we really like it, and think it should remain being the mapping agency for Britain because —

    MW: You see it goes to the point that you made early on about market failure, and all the rest of it; there are cases where you have to – I prefer to think of it as essential infrastructure that if you have – who is going to produce that mapping data? I’m told that in the United States there are whole swathes of that territory which isn’t properly mapped. Now, we don’t have that problem, and there are reasons why we don’t have it. Now, from going to providing what I would describe as essential infrastructure to saying that actually every bit of the way that infrastructure is realised and developed in the future, that’s a step too far in my view. Essential infrastructure, yes, anything else, well, you have to have a very strong case for government to go into that.

    MC: At the moment as you know OS’s public task is not defined, essential infrastructure is not defined, for all sorts of good reasons–

    MW: One of the things that I hope is going to come out of the study is that we do get clarity of purpose for the OS, It is something that is important, and it’s important – if we want these markets to flourish, and we do, and we want creativity and innovation to flourish, we have to know what the rules of the game are.

    Assistant: your next meeting….

    MW: Sorry – I can talk for a very long time about this..

    MC: The specific question about the resources available to OPSI – which it seems the government’s responses both to CUPI and to Tom [Steinberg’s] Power of Information report seem to dodge – both seem to be saying that the OPSI needs many more resources to police this market effectively. And this is your department…

    MW: It is my department, and —

    MC: You’ve already given it one more task [this morning] on the same budget–

    MW: They’re incredibly efficient, you see, this is the challenge of the marketplace, living within a budget, and they’ve done it brilliantly, risen to the challenge so far. They have to look – all departments always want more money, and believe me there are lots of good reasons to get more money in every department I’ve ever been in. We can’t make a case at the moment until we’ve seen actually what the territory is going to look like. The time to discuss all these things is next year. And we will be discussing it.

    CA: Thank you for your time.

    [Meeting ends: 32 minutes.]

    Departments weigh in on select committee Ordnance Survey enquiry

    Monday, July 23rd, 2007

    While we transcribe our meeting with Michael Wills – which was very interesting, but needs a proper transcript to do it justice and be sure it’s correct – here’s something to chew over. The Communities and Local Government select committee is holding an enquiry into Ordnance Survey, and ahead of any oral hearings has been gathering written evidence.

    And very interesting reading it makes – notably that from Defra, which is hardly the chummy stuff one expects from one government department about another (even if it is a trading fund):

    Defra and OS have enjoyed a close working relationship throughout the recent negotiations on INSPIRE. However, the Defra Network also experiences difficulties in sharing data derived from OS mapping with our wider delivery partners. [Emphasis added – CA]

    Difficulties, eh? There’s more:

    Defra co-ordinated and maintained the UK government position for INSPIRE. Officials worked closely with OS to safeguard the interests of Trading Funds. The EC starting point in negotiations had been that no charges should be allowed for licensing of data between public sector organisations.

    However, the Directive will require license terms and conditions for geographic data to be consistent across Europe and consistent with the objectives of the Directive, which are to support sharing and re-use of environmental data.

    OS mapping underpins a wide range of Defra Network activities including, for example, the administration of farming subsidy payments and the management of animal disease outbreaks.

    We also need to share data derived from OS mapping with our wider delivery partners, non-government organisations and the public. OS licence terms and conditions can constrain our ability to share this information. [Emphasis added – CA] It is our understanding that these difficulties arise at least in part from the dual role of OS as a public information holder and a commercially operating organisation, which is a specific area of interest for the Committee.

    There’s plenty more from other organisations. Any gems you’ve noticed?

    Environment Agency yanks flood data from OnOneMap site

    Saturday, June 30th, 2007

    We’ve written in the past about OnOneMap], which took the interesting step of taking the mobile phone mast data from Ofcom and making it available on its property/rental search site.

    But recently it did something much more interesting – and, given the weather, useful. At the start of June it added the Environment Agency’s flood risk data to its Google Maps implementation.

    The Environment Agency’s flood data is, to be honest, not as useful as it could be. You can do a postcode search, but when you then look at it, it is difficult to work out quite what’s at risk.

    An article in today’s Guardian – Agency’s flood maps fail to hold water – makes this point more elegantly. It appeared in the Money section of the paper.

    As the article points out,

    users will find the site lacks crucial details. For example, it fails to show the location of a home in relation to the area at risk of flooding.

    OnOneMap managed to grab the Environment Agency data (for England and Wales; we’re checking on Scotland) and added it as a layer to its Google Maps.

    The the Environment Agency got in touch: the data, it asserted, was its copyright, and it wasn’t happy about it being used in this way – even though OnOneMap is not (at present) a for-profit site. It asserted its ownership of database copyright in the data, which is hard to rebut, and threatened to take OnOneMap to court.

    (As I write this, the news is on, saying that the estimated costs of the flooding this month are £1 billion.)

    Without the resources to fight such a battle, OnOneMap removed the data – but not without making a note to that effect on its blog:

    The Environment Agency claims they have copyright over the information, and despite the fact that tax-payers’ money has paid for it to be collected in the first place, apparently the tax-payer cannot benefit from innovations like our housing and flood map combination.

    The comments on the post are quite illuminating, for example:

    This is absolutely outrageous given our tax money has paid for this. Surely it is up to the government agencies to ensure this information is widespread, ESPECIALLY during this time period when we are being inundated with water!!

    and

    Ridiculous, especially right now, that people need to find out if their home is at risk of flooding… greedy buggers, the Environment Agency… I loved the feature on ononemap.com, shame you had to withdraw it

    Interestingly, there is a site which does have very detailed data – which was gathered by the Norwich Union – is whatsmyfloodrisk.co.uk: you have to pay for the data (unsurprising, since it cost Norwich Union something like £5m to gather it..). But wouldn’t it be better if the Environment Agency data was available to all of us, free, without having to go to its site?

    In today’s Guardian: more examination of ‘The Power Of Information’

    Thursday, June 14th, 2007

    Today’s Guardian looks at the Ed Mayo/Tom Steinberg report The Power Of Information and asks what sort of Whitehall it would be that could open up, and what the effects will be.

    At the moment, the government’s attitude to the web is a mixture of aloofness and outright hostility. This should change, the report argues, partly because some of the most popular user-driven communities – MoneySavingExpert, for example – are closely linked to government policy. Others directly contribute to the public good: in Los Angeles, when the government started putting the results of food safety inspections online, the incidence of food-borne illnesses fell compared with that in neighbouring jurisdictions.

    The government should also be involved because online communities are big users of a repository of data generated by public bodies ranging from tide tables to school league tables. The internet greatly increases the value of this information. The humble postcode, originally developed for a single purpose, now underpins countless public, private and voluntary services.

    We’ve been around postcodes before, of course, but what’s interesting is that it has become a datum whose use has expanded far beyond its original intention by virtue of being mashed up with something else – geographical location. (In fact we might call postcode analysis the first mashup.)

    However, Government 2.0 is not yet official policy. The Cabinet Office will respond “in due course”, officials said, almost certainly to coincide with the government’s overdue response to the Office of Fair Trading’s report on the commercial use of public-sector information.

    As we said – long overdue. As in three months overdue. But it makes sense to lump these together, even if the Mayo/Steinberg report took only one-eighth the time to write by my estimate.

    The snag is that the response will need approval from arms of government whose income is likely to be hit by the proposals. If Ordnance Survey or the Meteorological Office had to give away information for which they charge today, they would look to their sponsor departments, Communities and Local Government and the Ministry of Defence, to fill the gap with tax revenues. With a tight three-year spending squeeze to be launched by the Comprehensive Spending Review in October, this would not be popular.

    Hmm.. but on the plus side, the report recommends that

    • By March next year, the government independently review the cost and benefits of supplying public information through trading funds. This would examine the five largest trading funds, the trade-off between revenue from sales of information, the wider economic benefits of giving the data away and the potential impact on the quality of data.

    • Public bodies, including trading funds, only to charge the marginal cost of distribution for raw information – which online is usually zero. The only exceptions should be where independent analysis shows that this does not serve the interests of citizens.

    • All trading funds consider introducing free licences for non-commercial re-use of PSI.

    • Ordnance Survey should launch its proposed OpenSpace scheme, allowing non-commercial users free access to data, by December. The service is currently on hold, the review says, because smaller commercial users object to data being made available freely to potential competitors.

    That’s got to be positive, surely.

    Ed Parsons, formerly of Ordnance Survey, now of Google

    Sunday, April 8th, 2007

    Ed Parsons, formerly the chief technology officer until last December at Ordnance Survey (whose reason for leaving was never made clear), now has a new job and a

    new office…:

    “This week I joined Google as the Geospatial Technologist for EMEA (Europe, Middle East and Africa) and to say I am excited would be somewhat of an understatement.”

    This is intriguing news, really, for OS. The launch last week of Google’s My Maps (maps.google.com), which lets anyone create their own annotated maps, means that Google is encroaching more and more on territory that the OS could once – before the web – have called its own.

    Now, OS will still have a grip on low-level mapping down to the metre-accuracy level (companies wanting to dig up streets need to be sure where buildings are) for now.. but if Google really sees a market in something, that might not be the case for long. And it might also be true that Google is repurchasing OS data.. but not all of it.

    Things are changing. And this is quite a change. Will OS now go ahead with its OpenSurvey project, which Parsons tried to make happen?

    Free Our Data: sessions this Thursday and Saturday, in Manchester and London

    Wednesday, March 14th, 2007

    The debate continues over free data: I’ll (Charles Arthur) be appearing this Thursday at Manchester University, with the National Centre for e-Social Science.

    The Manchester event is in the evening, starting at 6.30pm: I’m on a panel with Peter Elias, Professor of Labour Economy, University of Warwick and Strategic Advisor (Data Resources) to the ESRC; Duncan MacNiven, the Registrar General for Scotland; Jil Matheson, director of Census, Demographic and Regional Statistics at the Office for National Statistics; and Neil Ackroyd: Director of Data Collection and Management, Ordnance Survey.

    The topic:

    Should social researchers have unlimited access to the data that Government collects about individual UK citizens? Would the benefits of better evidence on which to base social policy outweigh concerns about privacy? Experts from academia and government will offer their views and respond to questions from the audience.

    With those attendees, it should be quite an event..

    On Saturday, I’ll be on a panel at 11am as part of the Open Knowledge Foundation conference in London

    with Ed Parsons, until recently the chief technology officer of Ordnance Survey, and Steve Coast, founder of OpenStreetMap.

    Read more on that, including directions etc, at the OScon site.

    Geology is free (well, will be)

    Friday, March 9th, 2007

    From the Guardian of March 9, Geological knowledge to go online:

    British scientists are leading an international effort to bring together all the known geological information about every country in the world. By making the data freely available and allowing researchers to track geological features across national boundaries, the project will make it easier to plan international projects, predict earthquakes and locate natural resources such as oil and gas.

    Once the project, called OneGeology, is up and running the data will be searchable via the internet. “Geology has no respect for national frontiers,” said Ian Jackson, who is coordinating the project for the British Geological Survey (BGS). “The data exists, but accessibility is the key.”

    We’ll be interested to see how it copes with saying precisely where these geological features are in the UK without reference to Ordnance Survey data; or perhaps it’s going to license it. In which case who’s going to fund it?

    I’m reliably told too that the Free Our Data campaign was mentioned at the launch – and that while Ireland is going to make all of its data at all scales completely free, the same won’t be true for the UK, where you’ll have to pay for data below a certain scale.

    That noise in the background is me, grinding my teeth.

    Hospital health mapping project blocked.. yes, Ordnance Survey again, this time vs Department of Health

    Monday, February 26th, 2007

    Yes, you’d rather read about how successfully we’re being in getting the government to listen to this campaign. (Well, we hope to have some more news on that later this week.)

    Meanwhile, you can read about how the North East Patient Health Observatory (Nepho) wanted to map data such as life expectancy, mortality and so on by local authority.

    Sorry, said the Ordnance Survey, but no. Hence the announcement:

    This service is currently suspended at the request of Ordnance Survey until an agreement for the use of certain data files and software has been reached between OS and the Department of Health. We regret we are unable to provide this service at this time.

    So have we got this right? One arm of government needs permission from another (very much smaller) arm of government to create maps of data that would help the very much bigger arm of government – and the taxpayers wanting to know what’s happened to their taxes in the bigger government department – find out what’s happening.

    Maybe it’s just me, but this seems daft.

    Update 1 March: Ordnance Survey’s PR has got in touch, and sends this response:

    NEPHO chose to withdraw from the pan-government agreement for mapping data when the Department for Communities asked them to pay a contribution like everyone else. They withdrew but continued to use unlicensed data. That situation is unfair to the other members of the agreement and we were required to ask them to stop. In order to find a solution we are actively working with NEPHO and advising them that they can access the data they need under the agreement as a contractor for the Department for Health. It is not a matter of us versus the Dept for Health as you claim in the headline.

    We’ll see if we can get some clarification from NEPHO.

    Phone mast data: it’s free, and out there

    Friday, February 23rd, 2007

    Interesting: ononemap, a property search company, has grabbed the data about mobile phone mast locations from Ofcom’s site and put it onto its own, along all the property ones.

    Free data; commercial organisation using it. Will this be an example of the effect of using free data in the private sector?

    It’s a very interesting application of (inevitably?) Google Maps, but the mashup – which as well as (obviously) properties for sale by area also optionally shows properties that have already sold, secondary schools, supermarkets and, now, mobile phone masts – which the sites notes at

    © Ofcom: Information reproduced accurately and in context from the Mobile Phone Base Station Database and with adherence to the spirit of the Freedom of Information Act 2000. OnOneMap displays the mast information as per the aforementioned database as of December 2006.

    .

    Is it bringing more visitors to ononemap? Is it helping house sales? (We feel obliged to point out that there is no evidence that phone masts have any effect on your health – though of course you might be able to bargain down a seller if you could point out there was one nearby…)

    In the Guardian: how Ordnance Survey keeps election boundaries to itself

    Friday, February 23rd, 2007

    In this week’s Guardian Technology, we look at the peculiar way that the Ordnance Survey treats the information collected by the Boundary Commission of England, under orders from the Electoral Commission (pay attention at the back..) when it’s drawing up constituency boundaries.

    Obviously the boundaries’ locations affects who might win elections in marginal seats; they’re being redrawn right now, in fact. Academics are interested to know how they’ve changed, in as much detail as possible, because they tell you interesting things about society, and so on.

    Turns out though that the Boundary Commission collects the geographic data and then puts its copyright on it. You can probably guess the rest of the story: those who want to use it for anything other than academic, educational or political purposes has to pay. Parties are banned from reusing the data (which OS makes available, for free viewing but no re-use, at election-maps.co.uk.

    However there’s an interesting – if limited – development:

    Last year, OS gave Professor Richard Topf, director for the Centre for Comparative European Survey Data based at London Metropolitan University, permission to show Westminster parliamentary boundaries for each election since 1983 (besis.org). Furthermore, it provided the data at no charge, encouraged its development and links to the resulting site from the Election Maps site. Topf says the reason the maps go back only to 1983 is simply because they are not available in digital form before then.

    But the site uses outlines of whole constituencies, rather than high-resolution mapping. Topf says OS required that high-resolution data could not be extracted from the site as a condition of its use: “It took several months, but that was because they wanted to know a lot about the technical construction of the files,” he says.

    OS says that agreements outside Edina have to be considered to ensure that material is used only for the purposes granted, and if it is for display use only, OS may require technical safeguards so data cannot be extracted for commercial use.

    Dr Southall says such agreements can be hard to come by: “The academic liaison side [of OS] is very, very helpful, but as soon as you say ‘copyright’, the whole organisation freezes,” he says.

    We heartily recommend besis, which is very interesting to play around with – though as the article points out, the detail is limited.

    But as for copyright freezing everything, see our post about whoownsscotland.