Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens


Archive for 2007

Ordnance Survey launches “OpenSpace” (sort of); Guardian reports on delays in government GIS report

Thursday, September 13th, 2007

Two related pieces of news.

First, the Ordnance Survey has launched – without any announcement we’ve spotted – an element of its OpenSpace platform, with the Explore portal. As originally conceived, this would have let people create mashups on OS maps. However, that functionality isn’t there yet; you can add “walks” (that seems to be all) to a 1:50,000 (Landranger-quality) map, though more is promised for the future.

Ed Parsons, formerly chief technology officer at OS, and now working for Google’s mapping division, comments on his blog:

Although this is nothing new – platial after all offered similar functionality a few years ago – this has been a long time coming. I was involved in some of the design work over a year ago! this is still an important step forward for the OS.

From a technology point of view the service was/is underpinned by the backend system developed to support the long delayed OpenSpace project, so hopefully there will be news about that soon.

Parsons concludes:

Although I would take issue with some of the T&C’s, this really is progress in the right direction from Southampton.

I’d not heard of Platial, but it certainly does do stuff that’s much the same (here’s a randomly-chosen “walk” in London), but using Google Maps.

The Explore page does show some Web 2.0-ness: it’s got a “blog” (more like comments) and shows the latest stuff. Except, as Parsons points out, any “walk” you submit becomes OS’s property. Eh?

Yes – here are the “portal rules“:

7.3 By submitting, posting or displaying any Submission on or through the OEP, User hereby grants to the Administrator a non-exclusive, perpetual, irrevocable, royalty free, worldwide licence to use, copy, edit, alter, reproduce, publish, distribute and/or sub-licence the whole and/or any part of the Submission, on or in connection with the OEP and for any other purpose.

In other words, OS can resell your stuff if everyone creates walks it likes. Not exclusively, but for itself.

Which in fact is exactly the same as we want the OS to provide its data to us, the citizens. Except that things seem to have gotten turned on their head, and OS is acting like a big media company such as MTV wanting to piggyback on the submissions of its viewers. (See “Whose content is it anyway?“, from Technology Guardian 21 September 2006)

Rather unhelpfully, although the page says that

The Administrator may update or revise the OEP Rules (including the Copyright / IPR Policy) at any time, with immediate effect, without notice. You are responsible for reviewing these pages regularly to ensure you are aware of any changes made and your continued use of the OEP after the changes have been posted means you agree to be legally bound by the new OEP Rules.

it doesn’t have a “last updated” tag.

Still, it’s movement, of sorts. But what we really want is an API so we can create mashups.

Update: Ed Parsons points out in the comments that it also bans links to “any page of the OEP Portal”. Let’s see how that one lasts online.

Meanwhile, Guardian Technology this week looks at the delays in the publication of an internal government report on geographical information:

Under the government-wide programme to transform public services through IT, a geographic information strategy for the UK was due to be published by July. But it has not yet appeared – and no publication date has yet been set.

Apart from the prime ministerial changeover, there is another reason for the delay: unhappiness that one organisation, Ordnance Survey, is both the government’s official adviser on geographical information and the main beneficiary of contracts to supply it. It’s roughly akin to Microsoft being appointed official adviser on government software purchasing.

The Association for Geographical Information has called for quicker publication of minutes of the government’s Geographic Information Panel; those from the June meeting haven’t appeared on the GIP’s website yet.

One final note of interest: this week’s edition of Guardian Technology carries a large recruitment advert on the front page – for staff in geographical information. It’s a burgeoning business…

Crown Copyright: time for an end?

Sunday, September 9th, 2007

In this week’s Guardian Technology we ask whether it’s time to end crown copyright. Time to take the jewels from the crown? examines the knotty question:

We argue that the UK government should follow the US in making all raw taxpayer-funded data available to the knowledge economy – except where that data compromises personal privacy or national security.

Some of our supporters say that a short cut to this state of affairs would be to abolish crown copyright itself. The idea is worth examining. Abolition was last floated in 1998, as part of a series of examinations in to what the government should do with its publishing arm, Her Majesty’s Stationery Office. A green paper, Crown Copyright in the Information Age (http://www.opsi.gov.uk/advice/crown-copyright

/crown-copyright-in-the-information-age.pdf, 273KB PDF), proposed abolition as one of seven options for crown copyright.

In the public consultation that followed, abolition emerged as the most popular. From 70 responses received, abolition received 12 votes as preferred choice. The runner-up (retaining copyright but in a simpler form) received eight votes. The snag was that although abolition was the most popular response, it was also the least popular, receiving the largest number of “unacceptable” votes.

Faced with this polarised response, the government chose compromise. In 1999, the Cabinet Office found a “general consensus” in favour of retaining copyright, with simplified procedures for re-use.

There are arguments for and against crown copyright – certainly, the one that we hear from ministers is that it would mean you could be sure that something did originate where it claims to have done. (A hash on the original data could be followed through and computed on subsequent data to check its origin, for example.)

But the problem with crown copyright as it stands, and more importantly as it’s used, is that it’s used to restrict. That’s what copyright was used to do originally (over the Bible: see the introduction given at the RSA/Free Our Data debate – transcript, as PDF – last year by David Vaver of the Oxford Intellectual Property Group) and still is. Even the word itself has that ring. Maybe we need a different word.

Bonus link: Wikipedia’s page on crown copyright

Government seeks input on flooding review: got an opinion?

Wednesday, September 5th, 2007

The Cabinet Office has set up a “Flooding Lessons Learned Review” site, where you – yes, you, the citizen in front of the computer screen – can comment in a helpful way, one hopes, to prevent this summer’s floods repeating.

The terms of reference says that the “specific objectives” are (emphasis added here):

  1. To understand why the flooding was so extensive.
  2. To learn lessons on how in future we can best predict, prevent or mitigate the scale and impact of flooding incidents in a potentially changing environment.
  3. To look at how best to co-ordinate the response to flooding in future, including the significant social implications for communities.
  4. To establish what access to support, equipment, facilities and information is needed by those involved in the response at local, regional and national levels.
  5. To ensure the public has as much access as possible to information on the risk of flooding to allow them to take appropriate precautions, be adequately informed on developments as an emergency unfolds, and be looked after properly in the immediate aftermath.
  6. To establish how the transition from response to recovery is best managed.
  7. To identify those aspects of the response that worked well and should be promoted and reinforced.
  8. To make recommendations in each of these areas to improve the UK’s preparedness for flooding events in the future.
  9. To make recommendations, drawing on the experience of the flooding incidents, to improve the UK’s broader ability to manage the loss of essential services in any future emergencies.

We’ve already commented on how the Environment Agency restricts access to its flood data – a fact that is complicated, we now learn, by the fact that although the EA is a government-appointed agency, its data is not crown copyright (because it may have to sue government departments, which are “owned” by the Crown, and the monarch can’t sue him/herself. Follow that?) So not only would the Free Our Data campaign have to get trading funds reversed, it would have to get agencies paid by government to put their data under crown copyright. Honestly, it’s one step forward and one back.

Notwithstanding, it would make a lot more sense if the flood map data was simply available to everyone to use and even improve upon. We’ll suggest that in the flood review. You’re welcome to add your own comments on the Flooding Review site.

A free, searchable law catalogue: hours of fun for all (especially lawyers)

Monday, September 3rd, 2007

The other day I was thinking to myself that one thing we could really do with is a database of court judgements. Not because I had any pressing need to look up a court judgement, but because logically – given that the courts are government-funded, work for the benefit of citizens, and are never ever going to be privatised – their output is a public property which should be available as a public good.

And lo and behold, someone has got there long before. Today’s Guardian leaders included one praising the work of Bailii, the British and Irish Legal Information Institute, a charity devoted to freeing the law and whose trustees are chaired by Lord Brooke.

The leader notes that

Since 1999, Bailii has been amassing past judgments and negotiating to publish current and future ones. This is not just a wheeze for lawyers to get something for nothing, nor is it only for people fighting their own cases through the courts. It provides a constitutional right, allowing anyone interested to read the decisions taken most days in the courts that influence the public domain. With a handful of people, a budget of just £120,000 a year, and a shrewd grasp of what technology can do, it is finally making a reality of the ancient quest for universal access to common law.

Having tried it very briefly and searched on a few points of interest, it’s clear that Bailii is trying to dom something very like what theyworkforyou.com does for Parliament – make the law courts’ output into something that we can all access. Its utility might not seem huge on face value, but that’s a false argument. Compare that to the utility you’d lose if it wasn’t there. Not sure what that is? Don’t worry – all databases grow in value with time and size.

The only shame, in fact, is that Bailii has to be charitably funded, outside government. It’s the sort of thing that we should be expecting the courts to do themselves. After all, £120,000 a year is about the cost of a few troops in Iraq. And I hear we’re withdrawing those now..

Want to put maps online cheaply? Get a paper licence and scan them

Thursday, August 30th, 2007

This will seem really strange, but tipped off by a comment on this blog from Walkhighlands, we investigated further. And it turns out to be true: with an Ordnance Survey “paper licence”, you can scan maps and put them online as you like.

Price difference compared to the digital licence: it’s about 1/50th price, or more if the digital licence waas quoted by a reseller.

There’s more at Paper maps rather than digital ones save site 99% in OS fees, which looks at the problems that Walkhighlands – an enterprising new small business based in Skye, which offers walks in those most beautiful (and remote, and in need of urban visitors looking to spend some time and cash, such as the picture above of Skye) – had in trying to put digital versions of the maps of its walks online.

Set up only this February, it operates from Staffin, in northern Skye, and already gets an average of 600,000 hits every month from about 18,000 visitors a month. It has recently been chosen as one of 26 new businesses to receive funding from Nesta, the National Endowment for Science, Technology and the Arts, and features more than 250 walking routes, which its organisers are expanding all the time.

But when Paul Webster, who runs the site, inquired into the costs of putting printable Ordnance Survey maps on the site (which already offers links to buy the full printed versions), he was horrified by the cost quoted by a reseller: £20,000 per year for the licence for the digital data. (OS says that it did not give a direct quote for that data). Trying to pay that would bankrupt the site.

..However…

instead, he turned to paper. “As long as you have purchased a ‘paper’ licence, you can scan maps and put them on the internet – as long as the webpage and the map doesn’t contain advertising,” he explains. He insists that the OS put this permission in writing – which it did, with a letter from a “senior service advisor”.

(We have seen a copy of the letter.)

This confirmed that “paper map extracts currently displayed on your website are covered by your Paper Map Copying Licence”. This, the letter says, is because “

  • the map extracts are being used as an information tool on your website to enhance your business
  • the mapping is not being sold and you are making no financial gain from the use
  • the map extract would need to be used in conjunction with the whole map sheet to give your extract context
  • there is no advertising on the same pages
  • “.

    Now, digital maps have far more utility than scans of paper ones – you can do all sorts of things with them that you simply can’t with a paper one, as Google and Streetmap and Multimap demonstrate. (You can’t dynamically calculate a route or a distance on a scanned map, unless you’ve done some very clever work locating corners.)

    But given that OS has to cover its costs, and that the paper licence only costs £50, it can’t be economical to issue the licence. It must cost more to administrate than is received. It would, surely, be cheaper and more efficient for OS to make paper map licences free.

    Which would be a start.

    Of course, as the article points out,

    this is not OS’s decision: ministers determine how it is funded. But with the trading fund model now being investigated, and OS under fire from a number of other government departments, the time is ripe for a radical revision of its funding regime.

    Time for ministers – perhaps with OS’s help, if it could say how much it costs to issue a paper licence – to have a think.

    (Afterthought: see, we do read your comments – and we do act.)

    Information World Review joins Free Our Data campaign

    Tuesday, August 28th, 2007

    We’re pleased to welcome our first official partner in the campaign: Information World Review, a VNU publication, has joined the campaign.

    In a posting on the IWR blog, IWR’s editor Mark Chillingworth notes that

    it would be great if Information World Review and its readers can be part of a campaign to make the information we already own more easily available.

    IWR – motto “Information for competitive advantage” – is Europe’s leading newspaper for the information industry, covering both content and information management issues from the perspective of information professionals and managers responsible for intranets, extranets, portals and content management. It is circulated to information professionals, information managers and content managers working in corporations, consultancies, and public sector organisations in the UK.

    As I note on the blog post, the more people and organisations we have on board, the better our chances of success. If you see more examples where government is charging or re-charging itself for data, let us know – add a comment here.

    Virtual London online plans killed off by Ordnance Survey licensing demands

    Friday, August 17th, 2007

    The map to the side was created by the Centre for Advanced Spatial Analysis at University College London, showing air pollution (redder means worse; bluer less bad). It’s the sort of map that would have all sorts of uses, if you could make it interactive and navvigable online: you’d be able to determine how polluted the street where you were looking to buy a house was; or (as a planner) where congestion was worst; or (as a scientist) where to site experiments.

    But you won’t find a navigable map like that – only snapshots. Reason being that longstanding attempts by Casa and Google to persuade the OS to license the use of the map online have foundered.

    The reason: Google wanted to make a one-off payment for the MasterMap data the map derives from; OS insists that it must be a per-user system, as applies to all sorts of other people (and which has scuppered other plans in the past, longstanding readers will recall – see “Travel maps of Britain.. measured by time, not distance” in May 2006. There too it was OS’s licensing model that meant that work funded by the Department of Transport couldn’t be shown online).

    In Want to see a great 3D model of London online? Ordnance Survey says no we look at what’s been lost:

    Virtual London, developed by the Centre for Advanced Spatial Analysis at University College, London, represents all of the capital’s boroughs in 3D, including 3m buildings. It was intended to help citizens visualise the impact of new developments and hazards such as air pollution and flooding. The mayor’s London Connects e-government programme has also sent copies of the model, running in Google Earth, to each of London’s 33 local councils.

    Then the problem emerged. Virtual London contains spatial data derived from OS’s MasterMap, the definitive crown copyright database of Britain. Licences to use MasterMap data are a valuable income stream to OS, a trading fund required to earn a profit for the Treasury by selling products and data licences. There was no problem with London’s boroughs using the 3D model in-house, because, like virtually all government bodies, they have licences to use OS data. What they could not do was post Virtual London on websites for London’s citizens to use.

    The Virtual London team blogs their disappointment:

    While it is fair to say that Google can be demanding the lack of movement by the OS does strike [smack? – CA] of an agency out of touch with today’s data requirements.

    The Free Data Campaign has a number of posts and information with regards the practices of the OS. While we have not always agreed with them, and indeed have been warned off openly criticising the OS in the past by the powers that be, we cannot deny that the whole episode has been slightly Pythoneque.

    The OS currently does not have the ability to license models for public usage and this is from a government-funded and approved agency.

    (Obviously OS would argue about the “government-funded” part of that last sentence. But since just under half its revenues come from licensing to central and local government, it’s at least partly correct.)

    The Virtual London team – while saying that they are merely passing on the link (to the world) while “worrying slightly in a ‘we need to distance ourselves from all this for the sake of our career[s]’ sort of way” – point to an article on the matter by the Londonist about OS entitled “Ordnance Survey are not our friends”:

    You’ve seen those adverts for a well known building society, right? – the ones with the annoying chap explaining that it ‘doesn’t work like that’.

    Change the building society for the Ordnance Survey (our national mapping agency) and make Google the customer for a farce that has made London the laughing stock of the mapping world.

    Google – Can we publish the Virtual London model from the guys at CASA? We’ll pay, and even put on your logo so that you get the credit.

    Ordnance Survey – Doesn’t work like that.

    Google – OK how does it work? Lets find a way around this, after all it is in the public’s interest and what with the Olympics coming up…

    Ordnance Survey – Doesn’t work like that.

    When you consider it like that, the whole thing really is Pythonesque. It could have come straight out of Life of Brian.

    Not waking the dead: time to make details of the dead available?

    Thursday, August 16th, 2007

    Overdue, but last week’s edition of Technology Guardian wrote about the place that we could call the dividing line between personal data and potentially free data: that of death. Like taxes, it’s inevitable. And notably under the Data Protection Directives, the dead do not enjoy data protection – details about a dead person can be passed without reference, because (obviously) they’re dead.

    That means though that dead people are (weirdly) potentially liable to identity theft (people assume the identity of the dead person, if organisations don’t realise they’re dead; infinitely less risk of the dead person spotting their credit record going sour, after all). And there are other problems too, such as people receiving mailshots at upsetting times, as the article – Direct mail reaches beyond the grave – explains:

    In June this year, the Ministry of Defence sent a recruitment mailshot to the family of Lance Corporal Dennis Brady, a reservist with the Royal Army Medical Corps, inviting him to re-enlist. But Brady had been killed eight months previously while serving in Basra.

    Although the blunder was very public, prompting a ministerial apology and the MoD to suspend direct mailing, it was not unusual. About 570,000 deaths occur in Britain every year.

    The key point is that death data should be shared within government (one of the drivers behind the e-government framework was one of its architects’ discovery that in trying to notify a death, a family had to contact official bodies 44 times over a period of 18 months.

    Releasing information about deaths, and the identities of the dead – so that they can be removed from databases (and so that credit and other applications in their names would be flagged) would obviously have huge benefits.

    Up to now, we have been concerned mainly with information about the natural environment and anonymised statistics, where we think the case for automatic free dissemination is clear. At the other end of the spectrum, data about identifiable citizens should be rigorously protected – the more so as the government moves towards joined-up databases.

    Records of deaths lie at the boundary between the two. We believe that, while the existing restrictions on the release of bulk data are absurd, the potential harm to living individuals means that re-users (including those in government) should be under a special obligation to get their facts right.

    We’ll offer a footnote here to the Bereavment Register, which offers a similar service, aiming to reduce the amount of direct (junk) mail sent to the deceased. It costs money, of course, whereas our scheme might make such prevention in effect free at source, since marketing databases could be cleaned against them. But a company like this could offer a service “cleaning” databases too.

    Administrative note: If you want to promote an organisation that has some connection to this campaign (or campaigns against it – we’re all for robust argument) then please use your real names. If you want to get in touch separately, then please email me – charles.arthur@gmail.com works well.

    More on government departments’ submissions to the Select Committee

    Thursday, August 2nd, 2007

    This week’s Guardian Technology supplement expands on the acid remarks from Defra and also the Ministry of Defence over the Ordance Survey’s licensing system in submissions to the Communities and Local Government select committee inquiry.

    In Ordnance Survey under fire from inside the government we note that

    the Ministry of Defence paints a picture of a 200-year-old relationship turning sour because OS has to operate as a business. While the agency is a world-class organisation providing an excellent service, “in recent times the boundaries applied to the use of OS’s data for public service and national interest work have become increasingly blurred. MoD has experienced more stringency and complexity being applied to the release of data by OS, which has resulted in uncertainty and lack of flexibility in the use of that data by the MoD.” Licensing charges set by OS are “particularly high”. As a result “some government users are being denied access”.

    Sound familiar?

    Also worth noting: there’s now a single page with all the Free Our Data stories from the Guardian. (It’s part of the website redesign. Maybe we’ll try on here some time.)

    Late, but anyway: New Zealand makes (some) statistics free – but why?

    Friday, July 27th, 2007

    Left over in the busy-ness last week was the piece in Guardian Technology about New Zealand’s government making a leap of faith and freeing a number of its statistical data that was previously charged-for.

    In New Zealand puts its trust in statistics, I ask Statistics NZ and Business NZ (the trade body lobbying on behalf of business) what the economic justification for the move was. The answer: nobody seems to know. But it seemed like a good idea.

    Interview with Michael Wills: full transcript

    Friday, July 27th, 2007

    The following is a transcript of the meeting with Michael Wills, minister for information, with representatives of the Free Our Data campaign. It’s been checked against the recording. There is a lot in here about government intentions if you read between the lines, particularly towards the end. Please do try to read it in full – and your comments are welcome.

    Interview with Michael Wills, minister for information, 19 July 2007.

    Present: Charles Arthur (editor, Guardian Technology); Michael Wills, minister for information; Michael Cross, Guardian Technology; a representative of the Office of Public Sector Information (OPSI); press officers and office staff for Michael Wills.

    Charles Arthur: Thanks, first of all, for setting up this meeting. We spoke to Baroness Ashton previously before the response to the [Office of Fair Trading’s] CUPI [Commercial Use of Public Information] report and the Power Of Information report; the responses were interesting. We really wanted to just talk to you about what our campaign aims to do and to some extent sound you out about what you think about where the use of government data goes, to put it into context with the freedom of information, we see the data side as being part of the same spectrum, but we’re very focussed on the impersonal side, which we see as having tremendous value when used commercially. The best example I think of a government project which provides data completely free which then generates economic benefits is GPS, which is a project that costs about $500m per year but according to evidence that’s been put forward in the House [of Commons] underpins the US economy; in the UK people use it for sat-nav that generates business, that generates taxes – so in effect we get a free ride from the US government. There are similar benefits in making data available for free for businesses to use because the businesses can then grow; if you give the data away free it might seem you’re forgoing all sorts of tax revenues but there’s a multiplier effect, that you then get a bigger economy and tax, and there’s a benefit….. and that’s our pitch.

    Michael Wills: Well, first of all thanks very much for coming in. Look, it’s a compelling pitch, that’s the first thing, and personally I’m very excited by this area, I asked to do this as part of my portfolio, when you become a minister … technically my title is the constitutional affairs minister. I wanted this – the Prime Minister appointed me to come and do the Constitutional affairs job, which is a big green paper that’s going to take a lot of my time, here and energy, but when I looked at the division of portfolios with the secretary of state I saw this area and I wanted to do it, I was previously responsible in one of my ministerial incarnations for FOI. The whole issue of data is I think tremendously exciting for all the reasons that you’ve said, it’s part of the infrastructure now of our society and our economy and it’s going to become more so with what’s happening with data mashing, the extraordinary intellectual creative energy that’s being unleashed is something that as a government we have to respond to, and the power of information you know is a very exciting document, something that I think is very much where government wants to be. Now, not in every detail, we have to work policy through very carefully, we have to be sure that what we do is going to work, is going to be effective, is going to be cost-effective and we will come back to some of those issues in a moment. But as a broad approach we are very sympathetic to that. I met Tom Steinberg last week to talk about his ideas of how we can move forward in a whole range of areas, it’s very exciting, and we have to respond, we want to respond but we have to–. So those are the key headline points, so overall we are very sympathetic.

    There are two issues really that we’ve got to address and they’re linked, I think– you stop me if you want to interject, but I think there are two issues we’ve got to address with this. One is the basic issue that all governments have to address in all areas of public policy: who pays, essentially. And as part of that we’ve got to be clear about what is the most economically efficient way of doing this. Now, in terms of the economic case, the case is on the face of it compelling, but compelling cases don’t always stand up to intense scrutiny. We can’t rush after our instincts on this; we have to have the job done carefully, properly, thoroughly, which we are doing. We’re committed to producing a study by the end of the year. We should be able to do that.

    Michael Cross: Do we know the terms of reference? Do we know who’s carrying it out?

    MW: I don’t know quite yet but we will deliver it by the end of the year, some things are commercially confidential, you understand but but but .. we have committed to carrying it out by the end of the – we will deliver this and in the end, look, we’ll be judged by this, this will be a public report, it will be open to everybody to asses, we will take views on it, we will be very clear about the openness of the consultation on the report and then we will make a judgement on where we go with it. There’s nothing, you know, secret about this, we have to, we have to make this case, whatever we decide to do, because whatever happens there will be someone will be paying the cost and whoever it is we have to justify that. Whether it’s the general taxpayer – because it is seen to be something that’s appropriate by the general taxpayer – partly because of all the arguments you’ve just made, but then we have to make the case for the general taxpayer; if the trading fund model persists we have to make the case to you and your campaigners and all the rest of it. But we will be judged by this report and we will engage with it properly. I don’t know what it will say, but it will be on the table and then we will move forward, and we’re not hanging around; as I say we should be ready by the end of the year. I know that can seem quite a long time in this world, but government–

    MC: Presumably you need to tender for the consultants to come in and carry it out–

    MW: They’re all being put out to public procurement–

    MC: The terms of reference–

    MW: In government terms this is moving pretty quickly.

    MC: What I was pressing for is the terms of reference, which by implication would have to be ready pretty imminently

    MW: We’re on course, OK. We’re really on course, we’re not hanging around on this because look, this world is changing so fast, and in a sense every year that goes by and we haven’t got this finalised one way or the other — it’s very, it’s a potentially very important decision, this is part of the infrastructure, and you’ve got to get this right. I think we’ve moved very positively already; we want to go on with that. And, now, just as an example of that, I just want to, OPSI are now, we’ve had discussions already about what we can do, they’re going to be setting up a web-based channel to gather and assess requests for public sector information–

    MC: –that was one of the recommendations of the Power Of Information report–

    MW: — and we would like you to become involved in shaping how we develop that. So I’d like to know if you would after this talk to [OPSI’s representative] and take that forward and – it’s not instead of, you understand, all the others things, but it’s a first step, that’s something we can do now. And how quickly [OPSI] do you think we can get this up?

    OPSI: We’re committed to July 2008, and from our point of view which is in response, we’re looking to do it as quickly as we can–

    MW: Could we do it quicker?

    OPSI: With a fair wind (MW: with a fair wind) – and it depends a little bit on what the community says that they want in terms of how exactly this thing works, because it’s really important to us to get it right.

    MC: A lot of work has been done in this area by the Demographics User Group and obviously Locus so we could come up with a user requirement, very quickly

    MW: That would be comprehensive as far as you were concerned, you reckon?

    MC: It would cover commercial use; we’d probably want something coming from the academic users, the association of geographical information–

    CA: And probably also the non-commercial users, individuals rather like Tom Steinberg started out as being, who have great ideas and expertise.

    MC: We could come up with a users’ manifesto pretty quickly I should think.

    MW: In a sense the quicker, the sooner we will move this.

    OPSI: As fast as we can.

    MW: But wanting to get everything done properly. We want to move forward on this.

    MC: You were saying there were two big issues – one was the affordability.

    MW: The other is the economic case; not the affordability, but who pays, which goes with the issue of the trading funds and all that; the world has changed dramatically since the 1970s [when trading funds were set up] and we have to reassess, that’s absolutely clear, but we can’t prejudge the study.

    MC: As you know there are a number of quite urgent competition concerns coming up – what can you do to make sure that these are addressed?

    MW: Well, they will be addressed. I have to address them; it’s not an option, we have to address them. I prefer to address them looking at this contextually and not be driven purely by reaction, which – we will react, we have to react, but it would be much better if we could have a comprehensive, holistic policy which also is flexible. This world is going to change again, so to some extent we have to, when we look forward, not get locked in. I don’t see why we should, I’m not saying we will, but — what I think we have to do, we have to be aware that this world is going to go on changing, probably faster than we think. And so we have to have an immediate response; we want to create a holistic approach next year. Realistically we’ll have the study ready; we will react very quickly to that, depending exactly what it says, and we can’t prejudge it, but depending exactly what it says, there’s no reason why we shouldn’t move very quickly–

    MC: One problem here is–

    MW: –our officials as you know are very good, very active, they want to get this right, so we should be able to move very quickly next year – [OPSI] you’ll tell me if I’m over-committing you all but we can move very fast once we get the study in place.

    OPSI: We have the [POI] recommendations, we have quite a tight timetable for reporting December 2007 to PA16? and then again in December 2008, and the machine is working bringing together the officials that are involved in that…

    MW: There’s no reason why that should slip at all is there?

    OPSI: None at all.

    MC: One complication though in this obviously is that we have the INSPIRE, the implementation of INSPIRE, has its own transpositions into UK legislation, its own timetable and that’s a complication you always get, isn’t it?

    MW: Well, it’s one of the things we’ll have to take into account, I’m afraid all public policy sector … competing, not competing, but differing inputs and we’ll have to manage that. I’m sorry, again, I’m not being evasive but I can’t be precise until we’ve seen what the study actually says.

    MC: Yess.. but we already have concerns, most startling I think in Defra and the MOD submission to the CLG [Communities and Local Government] select committee looking at the future of Ordnance Survey; in the context of INSPIRE it’s Defra’s submission which is actually quite forthright about the frustrations of the trading fund model in trying to implement INSPIRE, and these things that seem to require more urgent action than is coming from a holistic policy coming out next year.

    MW: More urgent than next year? Well..

    CA: To some extent, I think what the select committee submission gives Defra the chance to do is to make its feeling about Ordnance Survey public, and it gives it a forum in which to do it and gives it a reason why it should be doing it, but the sort of evidence that we’ve found in the 18 months or so since we’ve been doing the campaign is that there are all sorts of little bits within the public sector who are very annoyed — Ordnance Survey gets picked on a lot because it’s the biggest PSIH, the biggest trading fund, the biggest revenue–

    MW: And very successful–

    CA: And very successful, yes absolutely, within those terms of reference, but there are enormous frustrations at what people find they can’t do, There’s the North Eastern Public Health Observatory which wanted to do a map-based, in effect a mashup, of morbidity, mortality, various sorts of things, and found that they couldn’t, and that frustrated them and still frustrates them, it’s still up there as a big hole in what they want to do, and it comes to things like flood map data; floods, obviously, a billion and a half pounds gone down the drain in the past couple of months. The Environment Agency is in effect constrained from how it can licence its flood map data, partly because of the way it uses Ordnance Survey data; again and again we come across these sorts of frustrations with the fact that the trading fund model puts this sort of requirement on the trading fund itself.

    MW: Well, what you’re describing very graphically is the opportunities that are there. That’s why we’ve got to move, that’s why the government are so positive about this agenda, you’re just proving again the point that there are these fantastic opportunities. Look at it slightly differently, from the point of view of government we accept the opportunity, the case for moving on this, we’ve got to, and we are, already. Government inevitably, and rightly, moves quite slowly and cautiously, we’re dealing on behalf of all the people we serve, and you can’t be rash about this. In this particular area government is moving, for government, very fast precisely because we do all see the opportunities that you just described. They are there. And what’s so exciting about all this is that you know that even though you can’t specify all these things, once you let this creative energy go, people will come up with extraordinary things out of this. None of us around this table could predict it will be, or exactly when, but we know that in the next 5 to 10 years extraordinary things will come out if we can unleash this creative energy in the right way, so the point is absolutely taken, there’s no question about that, and we will move as quickly as we can. But, having said that, from looking at the need to move, develop, find the right way to unleash all this energy, to actually what the model should be, is – you can’t do it in one step. We have got to just take a little bit of time, and I mean a little bit of time, we’re not pushing it into the long grass, they are very tight timescales, the study will be available at the end of this year… this is rapid.

    MC: Do you concede that both CUPI’s report and our anecdotal evidence suggests there is something seriously wrong with the trading fund model as it now works in the information market?

    MW: Well I prefer not to put it like that, what I would prefer to say is that there are huge opportunities out there and we have to be certain that we’ve got the right way of realising them.

    MC: Of course, yes–

    MW: That’s the way you’ve got to look forward–

    MC: But you’d concede there’s prima facie that there are a lot of problems with the current model.

    CA: In effect a market failure.

    MW: We admit, well, we think that there’s prima facie evidence that we need to look at this again, that’s what we would say, I think. Of course we do. There’s no question – the trading fund model was established 30-odd years ago. It would be extremely foolish of us not to look at it again, which we are.

    CA: Will the – I couldn’t quite work out whether the TOR have been drawn up or are being drawn up, will the examination of the trading funds look at other countries’ experiences or will it be limited solely to the UK trying to determine whether it’s efficient or inefficient in the UK?

    MW: Other countries’ experiences are always interesting – you’ve got a piece today on New Zealand, which is interesting – but they’re all slightly different, and there are all sorts of reasons why this will have to focus on this country. One of the reasons why we’ve just — it’s important to remember that we have very high quality public sector data in this country and the trading fund model though it was set up a long time ago has clearly been doing something right in that way. The reason that this is so exciting now is partly because we actually have such riches here produced under the trading fund model. Now, that is not an argument for no change, please don’t misinterpret that, but it is saying that we just have to be careful that as and when we change, that we don’t lose some things that are precious and — I could give you, but I won’t, a defence of the trading fund model because I think we’re now looking at all this again, but we have to recognise that however, whenever it was formulated it has produced something very important. Now, the world is completely different now and we can’t just assume that because something worked for the last 30 years it’s going to work for the next 30 years, it won’t, it will not, and therefore we have to change, and that’s why I’m slightly careful. Now the economic case that is produced is just part of the case; we will be guided by it, obviously, that’s why it’s been commissioned, but it won’t be definitive; I just say that, we will listen very carefully to the responses that we get; it will be one study, it will be an important study, but we will be listening to the comments on it and the discussion; we have to be sure that we’re getting this right, we’ll only do that by consulting and engaging and assessing the responses fully, openly.

    MC: The other question apart from the TOR is picking who’s going to carry it out; you can imaging in a small world of experts.. it’s a sort of snakepit of jealousies…

    MW: It’s not just in this sector (laughs). As I say, we have to get this right; that is our criterion. I can assure you I have no set view on what the outcome should be. Honestly, I want to see the study, I want to hear the responses to the study, then want to decide. But the responses are going to be as important as the study itself, if that’s any reassurance to you. I understand why you’re pushing on this, and you’re absolutely right to do so; there are various things I can say now and various things I can’t quite say right at the moment.

    MC: Who will actually appoint the – what shall we call them, the reviewers.

    MW: Government.

    MC: Government. Will – you have the resources of APPSI–

    MW: I’m the minister responsible for this; there are a number of government departments which have a keen interest in this, and we have to work with them. We have to be very clear that we’re setting up a model that is going to be sustainable. The worst thing that we could do right now would be to set up a model that we would have to tear up and do again in two years, because that doesn’t provide the stability. If we’re looking at economic benefit, the one crucial thing is that the model has to be sustainable. Otherwise you won’t get the investment, you won’t get the energy; people are, investors particularly, always wondering if this [business] is going to be sustainable. You won’t get the venture capital – that is not indispensable – but as we know from other areas in this world, if you get creative risk-taking venture capital involved, then that is a crucial factor in unleashing creativity. So we have to have a sustainable model, and that means we have to take Whitehall as a whole with us and [it] has to be signed up to generally. Although I’m responsible.

    CA: So does that mean that Treasury has an input in terms of how–

    MW: I think you’ll find Treasury has an input into everything in Whitehall.

    MC: To move on, what’s your feelings about where the public sector should be involved in the knowledge economy? One of the things we – perhaps it’s a personal hobbyhorse – is a website called Transport Direct–

    MW: Oh, yes.

    MC: –which is a classic government effort, technically it’s probably brilliant, though there are all sorts of things about it routing wrong – it’s the classic all-singing all-dancing, expensive, probably available in Welsh, government work project but it squeezed a lot of private sector people doing this sort of stuff out of the market and has a chilling effect on the market for travel information ever since. Do you think government is right to be doing this sort of thing?

    MW: I’m not going to comment on Transport Direct because I don’t know about it. What I’m about to say should not be said to be about Transport Direct, and I can’t make a judgement. Look – government has to be involved, and is involved, because for all sorts of reasons, in this area particularly it generates huge amounts of data; it has to – and therefore you can’t remove government out of it. But I think we have to be extremely careful about the role that government plays in markets, and there is always a risk that the state can stifle innovation and creativity; it can be innovative and creative in its own way and can produce things, but — you probably don’t know, but I used to run my own business in television, so I have operated in that sort of intensively competitive marketplace for longer than I have as a politician, and not surprisingly I believe in its virtues. I operated in a sector when there were lots and lots of really quite small, highly creative, highly professional companies, all fighting each other for really quite a relatively small market, intensively competitive. And perhaps – you’ll forgive my personal bias – but I thought it worked quite well you get a lot creativity, television in this country is constantly renewing itself, but it is a highly creative industry which attracts lots and lots of very creative people, which has gone on doing that for generation after generation and it works, and the explosion of activity that took place in the 1980s and 1990s, after liberalisation of the market, I thought was very productive, So – and I don’t think that, but — having said that in this area, that I do know well, I think the BBC remains having a fundamental role; it has to, and it goes back to what I was saying about the infrastructure, there is always going to be a role for government in providing some sort of infrastructure.

    MC: There is a role for government–

    MW: You can’t remove government’s role in this particular area because we are a player, and cannot help but be so; there is also a positive role in providing the infrastructure, which we’re doing, which all this is about, providing the proper infrastructure so all this can take place, But beyond that, I think you have to go with very great care as government. By and large the presumption must always be is that you let people go and follow their own instincts, let them make some money; if it works – great. That’s the way you’re going to get the creativity and the energy and the unexpected consequences, you can’t do these things top down; everything tells us that if you let people just get on with it something wonderful is going to happen. If you start saying what it is that’s wonderful that should happen, you won’t get something wonderful.

    MC: So given that would you accept that over the past, say, five to seven years for all sorts of good reasons government has perhaps moved too far up the value chain in PSI?

    MW: Well, I could tell you – no, I won’t tell you the story. I’ve got a lot of very young officials around the table who have long lives ahead of them and I don’t want to stop them by giving them a heart attack right now. I’m not going to tell. But, I think we have to be very careful as government; any official body has to be very careful about entering into a marketplace. We have to be exactly clear, scrupulously clear about the reasons why we’re doing it and we should never just go into it because we think it’s a good idea or might be interesting; and by being clear about it – what we can contribute to it, that we’re not going to stifle creativity and innovation coming up from the grassroots.

    CA: Our argument isn’t and has never been that OS should be sold off or should be hived off in any way – our argument is that Ordnance Survey is a terrific organisation that does great work, and we really like it, and think it should remain being the mapping agency for Britain because —

    MW: You see it goes to the point that you made early on about market failure, and all the rest of it; there are cases where you have to – I prefer to think of it as essential infrastructure that if you have – who is going to produce that mapping data? I’m told that in the United States there are whole swathes of that territory which isn’t properly mapped. Now, we don’t have that problem, and there are reasons why we don’t have it. Now, from going to providing what I would describe as essential infrastructure to saying that actually every bit of the way that infrastructure is realised and developed in the future, that’s a step too far in my view. Essential infrastructure, yes, anything else, well, you have to have a very strong case for government to go into that.

    MC: At the moment as you know OS’s public task is not defined, essential infrastructure is not defined, for all sorts of good reasons–

    MW: One of the things that I hope is going to come out of the study is that we do get clarity of purpose for the OS, It is something that is important, and it’s important – if we want these markets to flourish, and we do, and we want creativity and innovation to flourish, we have to know what the rules of the game are.

    Assistant: your next meeting….

    MW: Sorry – I can talk for a very long time about this..

    MC: The specific question about the resources available to OPSI – which it seems the government’s responses both to CUPI and to Tom [Steinberg’s] Power of Information report seem to dodge – both seem to be saying that the OPSI needs many more resources to police this market effectively. And this is your department…

    MW: It is my department, and —

    MC: You’ve already given it one more task [this morning] on the same budget–

    MW: They’re incredibly efficient, you see, this is the challenge of the marketplace, living within a budget, and they’ve done it brilliantly, risen to the challenge so far. They have to look – all departments always want more money, and believe me there are lots of good reasons to get more money in every department I’ve ever been in. We can’t make a case at the moment until we’ve seen actually what the territory is going to look like. The time to discuss all these things is next year. And we will be discussing it.

    CA: Thank you for your time.

    [Meeting ends: 32 minutes.]

    Government asks Free Our Data to work with OPSI on web channel for users

    Thursday, July 26th, 2007

    We did say that the meeting with Michael Wills was interesting. In today’s Guardian, we precis the meeting (update: the full text is now on the blog), which boils down to a few key points:

    • Michael Wills, the minister for information at the Department for Justice, thinks the case for free re-use of public sector data in and outside government is “compelling”
    • He thinks that government has to move quickly to adjust to the changing information world
    • He expects the ‘public task’ of Ordnance Survey to be defined within a study that will report to him by the end of this year
    • the trading fund model, set up in the 1970s (by which government agencies charge for their output, reducing their need for direct tax funding) needs reexamination
    • He is reluctant, however, to disturb a model (trading funds) which has produced high-quiality information

    They key point though is that he wants Free Our Data to work with the Office of Public Sector information to set up a web channel through which the public can request public data, and what form they want it. That, to us, marks a significant recognition of the importance of this campaign.

    Read more at The minister will hear you now in the Guardian. The full transcript of the meeting will be posted once it has been re-checked for accuracy.

    Departments weigh in on select committee Ordnance Survey enquiry

    Monday, July 23rd, 2007

    While we transcribe our meeting with Michael Wills – which was very interesting, but needs a proper transcript to do it justice and be sure it’s correct – here’s something to chew over. The Communities and Local Government select committee is holding an enquiry into Ordnance Survey, and ahead of any oral hearings has been gathering written evidence.

    And very interesting reading it makes – notably that from Defra, which is hardly the chummy stuff one expects from one government department about another (even if it is a trading fund):

    Defra and OS have enjoyed a close working relationship throughout the recent negotiations on INSPIRE. However, the Defra Network also experiences difficulties in sharing data derived from OS mapping with our wider delivery partners. [Emphasis added – CA]

    Difficulties, eh? There’s more:

    Defra co-ordinated and maintained the UK government position for INSPIRE. Officials worked closely with OS to safeguard the interests of Trading Funds. The EC starting point in negotiations had been that no charges should be allowed for licensing of data between public sector organisations.

    However, the Directive will require license terms and conditions for geographic data to be consistent across Europe and consistent with the objectives of the Directive, which are to support sharing and re-use of environmental data.

    OS mapping underpins a wide range of Defra Network activities including, for example, the administration of farming subsidy payments and the management of animal disease outbreaks.

    We also need to share data derived from OS mapping with our wider delivery partners, non-government organisations and the public. OS licence terms and conditions can constrain our ability to share this information. [Emphasis added – CA] It is our understanding that these difficulties arise at least in part from the dual role of OS as a public information holder and a commercially operating organisation, which is a specific area of interest for the Committee.

    There’s plenty more from other organisations. Any gems you’ve noticed?

    We’re going to see the minister..

    Saturday, July 14th, 2007

    Perhaps the Free Our Data concept is getting some consideration in ministers’ minds. One of the first thing that Michael Wills did on being promoted to his new job at the Department of Justice was to ring us up and ask for a meeting.

    Eh? Ministers calling journalists? Quite a turnaround. Not even his private office, but the man himself.

    As a result we’re going to see him later this week, to make our case formally – and, perhaps, to see if there’s any formal response beyond that on the OFT’s Commercial Use of Public Information report, and perhaps the Cabinet Office response on the Power of Information.

    Among the examples we’re planning to use re government making things free providing a boost to commercial sectors:

    • GPS: the US government-funded Global Positioning System costs about $400m to run per year, but underpins a huge business reliant on its positioning data
    • museums: the Labour administration made admission to museums free. Why? To encourage more people to go to them. (The Conservatives propose to reverse it.. perhaps) It has led to more visits (though Hazel Blears points out that it’s not all been encouraging .. but a letter in June from senior museum directors says they like it – but that it needs funding. (Has anyone analysed the economic effects of 30m extra visits to museums – as in the train and bus journeys, the gifts bought, and so on?)
    • the examples of South Africa, Canada and New Zealand in making at least some of their data free

    Anyone else with the sort of arguments that would help a minister persuade the Treasury to unzip the purse strings and give the information economy an extra boost? Your suggestions – with sources, please – welcome.

    Who’s who after the reshuffle

    Thursday, July 12th, 2007

    Today’s Guardian asks “Can the new ministers make a difference?” and looks at who is now who following the reshuffle by Gordon Brown.

    Key players now are:

    • Michael Wills (in charge of OPSI and the National Archives, and Freedom of Information)
    • Stephen Timms, now minister of state for competitiveness, will have a key role in setting the rules under which trading funds such as the Meteorological Office and Ordnance Survey operate in the information market
    • Derek Twigg at the MoD (which has some say over the UK Hydrographic Office and Met Office)
    • Lord (Jeff) Rooker at Defra, who will have to oversee the implementation of Inspire
    • Baroness Andrews at DCLG, which includes overseeing Ordnance Survey
    • Ed Miliband at Cabinet Office, who’ll have to look after the web 2.0 and the implementation of the ideas accepted from The Power Of Information report.

    We’ve linked to their Parliamentary profiles on TheyWorkForYou so you have some idea of what they’ve been interested in previously… though of course that’s not necessarily a guide to how they’ll act as ministers.

    We’ll have an announcement regarding Michael Wills in a later post.