Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens

Archive for the 'Media coverage' Category

Show Us A Better Way: the winners are chosen

Thursday, November 6th, 2008

It seems like an age ago that Tom Watson and the Cabinet Office kicked off the Show Us A Better Way competition, intended to see whether there really was an entrepreneurial spirit out there that wanted to get hold of government data and make something with it.

It turns out that yes, there was. And like a team of Bob the Mashup Builders, many people said “Yes, we can!”

And now the competition has got its results. (And the Guardian reports it exclusively.)

Tom Watson, the Cabinet Office minister who pushed so hard to get it off the ground, said: “These excellent ideas are born out a truly democratic competition which has seen entries submitted for all over the world. Show Us A Better Way has really captured the imagination of people in their own communities. They are telling us what information they want and how they want to use it.

“I have been delighted by the ideas submitted and how ingenious people have been in applying the information that government already holds. This is about taking service design out of Whitehall and to the people who use it. I hope the people behind the ideas that just missed the cut will not be disheartened and will continue to develop them into working websites.”

So who are the winners? In no particular order, in the first category they are:

  • Can I Recycle It? Input your postcode to find out what your council recycles
  • UK Cycling A one-stop site to plan your cycling route, for those at any skill level
  • Catchment Areas Shows boundaries of school catchment areas, even “fuzzily”
  • Location of Postboxes Shows where the nearest one is to wherever you are
  • Loofinder A mobile texting or website that tells you where the nearest public toilet is.

But there are two other categories:
Ideas where the government will develop the idea further:

  • Road Works API, an interface to any and all roadworks so that organisations (such as satnav companies) or individuals could build alert systems;
  • Oldienet, which would tell you about services in your area;
  • Free Legal Web, which would be an authoritative mashup of expert legal commentary and public-sector information;
  • Allotment Manager, for better allocation of garden allotments; and
  • Where Does My Money Go, an interactive web application showing government budget data via maps, timelines, graphs and charts.

There were then another four that were declared to be “fully working” already, as prototypes, and which will be funded for further development:

  • UK School Maps (showing where the UK’s schools are – building on data released for the competition by the Department for Children, Schools and Families);
  • School Guru, which helps determine whether your child could get into a school (in Hertfordshire only at present);
  • Where’s the Path, with an Ordnance Survey map and Google Maps satellite picture of any spot; andthe
  • UK Wreck Map, showing the location of undersea wrecks around Britain’s coast.

The judging was difficult, and protracted. You may wonder why the ideas that got the largest number of votes on the Uservoice site didn’t automatically get the prizes? Because we recognised that there were weaknesses in the Uservoice system – one could vote without registering, or just wipe your cookies and vote multiple times – and also that the aim of the competition was to reward ideas that were truly innovative, that would stretch the capabilities of government data, and not just replicate services that already existed either within government or commercially (a surprising number of entries did, one way or another).

Thanks of course to the DCLG (£40,000), Ministry of Justice (£20,000) and Cabinet Office (£20,000), which all contributed to the prize fund.

But that’s not all: the final winner will be announced on BBC’s iPM programme on Radio 4 this Saturday. Let’s see who gets it.

Today in the Guardian: ‘free data’ ministers still in place, but face uphill challenge

Thursday, October 9th, 2008

The dust has settled from the ministerial reshuffle of last week, and we’re happy to see that the ministers whose views about access to government data chime with ours – particularly Tom Watson in the Cabinet Office and Michael Wills at the Ministry of Justice – remain in place. Note too that Shriti Vadera, who was at DBERR, now has a higher-profile role which is expected to have quite an impact. (Baroness Vadera, a former City high-flier – remember them? – is understood to hold strong views on, inter alia, making government data more easily accessible.)

In today’s Guardian we note the fact that that hasn’t been shuffled around, and the new challenge that ministers pushing the free data idea face: how do you persuade a government that has just melted down the golden rule in order to quasi-nationalise high street banks at a cost of around £500bn, with what looks like a shrinking economy on the way, that it should forgo hundreds of millions of pounds in tax funding to pay to make data free?

(One answer might be: because it’s cheaper to do that than pay the unemployment benefits and other consequent costs if companies that pay for government data go bust.)

In Free data faces a tough challenge in the new parliamentary season, Michael Cross notes that

Decisions about the future of such trading funds will need to be made soon. A much-delayed government-wide strategy for geographical information is due for publication this autumn. It has spent the past year being bounced between Civil Service desks as its ideas are aligned with Britain’s commitments to open access to environmental data under the EU Inspire Directive. More interesting for the Free Our Data campaign will be the outcome of a review by the government’s Shareholder Executive into the trading fund model.

Oh, yes, that review. Today’s bonus link: Parliamentary Question from the Tories’ shadow minister for innovation and skills, Adam Afriyie: who has the review team met?

The answer:

the Shareholder Executive team has heard the views of around 20-25 stakeholders from the private sector, as well as others from the public and third sectors. The private sector stakeholders have included customers, suppliers and competitors of the trading funds, small UK-based companies, large multinationals and representatives of trade associations and interest groups.


It would not be appropriate to name the organisations individually because their contributions to and comments on the assessment have been to inform advice to Ministers and were on a non-attributable basis.

Vote for the idea you think should win Show Us A Better Way

Thursday, October 2nd, 2008

The government’s Show Us A Better Way competition has finished its first part – getting entries. And there are lots of them. (More than 500, by a rough count.)

We’ve been asked to help draw up the shortlist, and take part in the judging panel. And we’ve been specifically asked to get everyone out there to vote on their preferred ones.

You can: go to and get stuck in. Almost all of the entries are there, apart from some last-minute entries. (There may also be duplications: when you find them, leave a comment and I’ll try to fix it.)

You each have 10 votes; use them wisely.

How should you choose? Ah, yes, that’s the question. I think that ideas that truly deserve the £20,000 prize should be (1) widely useful [which I think rules out applications that would only run on particular computers or phones] (2) achievable [ie not requiring supercomputers, or everyone installing some custom-made widget] (3) beneficial (4) not duplicating something that could be or is already done commercially (5) has that magic something which makes you think “ooh, clever!”.

Those, anyway, would be my suggestions. You’re welcome to use your own criteria. But get voting and tell your friends!

We’ve also written about it in the Guardian. Even so, tell more people..

FOD interviewed for BBC iPM on making court records available online

Sunday, September 28th, 2008

We were happy to be interviewed for the BBC’s iPM programme (Radio 4, Saturdays, 5.30pm) on the topic of Jack Straw’s announcement that court records will be made available online.

Now, the Free Our Data campaign is, strictly speaking, about non-personal data: we argue that should be made available for free re-use. When you’re talking about court records, that’s rather different: it’s about as personal as you can make it.

But there is a wider principle, which is that it seems to us good if the government is wrapping its collective head around the idea that data can be useful, and that the assumption should be that data are made available, rather than kept secret.

Some lawyers argue in the piece that the court records are riddled with inaccuracies. Obviously, that would have to be ironed out.

But there’s a wider point: newspapers now have online archives, and they don’t delete them. (It’s a principle at The Guardian, for example, that we don’t change what’s on the site without very good reason.) That means that these records are going to be there, even if the government doesn’t make them available.

To be honest, Lord Falconer’s argument that articles relating to high-profile court cases should be removed from online news archives because of the risk of prejudicing trials is simply untenable. It won’t happen. Bloggers will write things. American sites will collect data. Google’s cache holds data. The Wayback Machine holds the data. Once on the internet, data tends to survive. Falconer’s suggestion is typical of people who can’t conceive that the internet has changed the idea of access to data completely.

My suggestion is that, given these facts, we simply need to move to a situation like the US: where juries are sequestered, and told to try people only on the facts of the case (as they are here too, of course).

Of course besides making it easier to retain data, the internet makes it easier to spread it – and, potentially, create businesses from it. Which is also what this campaign is about.

Welcome, iPM listeners

Saturday, September 27th, 2008

If you’ve just come here after the iPM item – welcome. Do have a look around. We have studies of the economics of freeing data and Ordnance Survey’s lobbying against our case and all sorts..

We’d suggest that once you have, you take a look at the clearest fruit of our efforts – the government’s Show Us A Better Way competition, where people are suggesting ways to use government (non-personal) data to create new services. Not long until the closing date..

Ordnance Survey’s lobbying, part 2

Thursday, August 28th, 2008

In Guardian Technology of August 21 we reported on Ordnance Survey’s hiring of a lobbying company called Mandate, and how it had kept watch on MPs and organisations which seemed to be interested in the whole “free data” concept.

Ordnance Survey responded to the story: this is a reprint in full of its letter. Following this, how a story in today’s Guardian Technology examines the contradictions between what OS says it does (and what its minister, Iain Wright, responded in his original Parliamentary answer – that it’s “consultancy and advice on Corporate Communications and Public Affairs”) and what the email track seems to suggest.

First, the letter:
“We are more than happy that the Guardian has shown this interest in the way that Ordnance Survey communicates about the important work that we do.
“Ordnance Survey data helps underpin life in Britain. It is relied on by business and society, from battling the effects of climate change to the sat nav in millions of cars. Our data is mapped down to the nearest few centimeters and updated up to 5000 times a day. It is this consistent level of quality, currency and detail that makes it so vital for public services, ranging from emergency planning to the delivery of everyday services on the ground.
“It is because Ordnance Survey data is so vital that parliamentarians and other important stakeholders expect us to communicate with them about our work. That is why we engage with politicians from all parties who care about the services that we provide. We have a duty to inform them on our role collecting the data needed to map every feature on the landscape, and how we intend to maintain the quality of this sophisticated data going forward.
“We’re committed to the best possible communications with all our stakeholders, now and in the future.
“Nicole Perry head of public affairs, Ordnance Survey”

And so to today’s story, Ordnance Survey defends its use of lobbying company:

The Free Our Data campaign agrees with Perry that the need to educate opinion-makers about geographical data in the digital age is an important part of its public task. However, a study of the 361 printed pages of correspondence between OS and Mandate, obtained under the Freedom of Information Act, suggests that its publicity campaign strays into broader areas of government policy. In particular, on the question of whether it and other information agencies should continue to be run as trading funds, required to cover their costs by charging for access to data. (The Free Our Data campaign argues that this policy hampers state and community initiatives to make better use of data.)

Among the papers we received is an account of a seminar on trading funds, organised on April 29 by Locus, an industry body which represents users of public sector information (and which retains its own lobbyist, a firm called Quintus Public Affairs). In an email to Perry, a Mandate executive reveals that she attended the seminar, accompanied by a colleague “Eleanor”, and reports back “on comments from the meeting that you should be aware of”. These include the views of Locus’s chairman, “Bryan Carlsberg” (sic – his name is Carsberg) that member companies should talk to the Conservative party on this issue “as they are currently looking for proposals for their manifesto”.

We’re just trying to square that with Iain Wright’s suggestion that this is “consultancy and advice on Corporate Communications and Public Affairs”. It seems very like, well, straying into areas of policy.

And there’s also the question of quite what is recommended.

On April 24 this year, Mandate alerted Perry that a Conservative MP, Greg Clark, had tabled a question about the relationship with Mandate. The email urged Perry to “please rest assured” that Clark had asked many such questions, and that the information needed in response is “minimal”. We will see whether Ordnance Survey’s minister follows that advice.

I had always thought that it was the responsibility of departments and government agencies to seek to answer Parliamentary Questions as fully as possible; if this is not done and the minister answering is not sufficiently briefed, it can be extremely embarrassing, initially for the minister. Is “minimal” advice sufficient? And overall, has Mandate really been good value for money?

One other thing we’ve heard:

Ordnance Survey’s use of a lobby firm to engage in the free data debate is likely to be on the agenda at the next meeting of the government’s Advisory Panel on Public Sector Information next month.

We’ll look forward to the minutes of that meeting.

Interactive crime maps for everyone by Christmas, says Home Office

Monday, July 28th, 2008

Despite the fact that Parliament has risen (so that it’s officially the silly season – hey, was that a UFO flying past?), the Home Office is still busy at it. Today, it’s put out a press release saying that

Every neighbourhood in England and Wales will have access to the latest local crime information through new interactive crime maps, Home Secretary Jacqui Smith announced today.

The rollout of interactive crime maps follows the announcement made by the Home Secretary earlier this month, as part of the Policing Green Paper, that every police force in the country has now delivered monthly crime information to the public on their websites. New interactive crime maps will take the rollout of local crime information to the next level.

By the end of the year every police force area will produce crime maps which will allow the public to:

* see where and when crime has happened, down to street level for some crimes;

* make comparisons with other areas; and

* learn how crime is being tackled by their local neighbourhood policing team.

We think that the last of those is going to be very interesting indeed, since for senior police officers it will (in a nice phrase I heard on a related topic from a civil servant recently) “hold their feet to the fire”. (Strange how one has to summon images of torture when trying to get some public services to change..)

Coincidentally, we’ve had some interesting emails on the topic: one from (which, you’ll find, does some crime mapping).

And another reader wrote in at length:

“About 3 – 4 years ago I worked temporarily in Bury MBC’s Housing Department. There was a man in the Chief Executive’s department who had a GIS containing 3-years-worth of police crime data. He could rustle you up a map of recorded crimes, varying by type and date, for any local area you chose, on request. So it can’t be that difficult to do it.

“In the early 1970’s I was Area Housing Manager at Speke in Liverpool. My office was in the middle of this Council-built area some 6,000 houses and flats and the local police station was just across the street. This was long before we had computers for anything except (batch processed) rent accounting and it was before “defensible space” became an idea in good currency amongst urban designers.

“Following a disturbing interview with a widow with three children whose chronic poverty had been made even worse by being burgled 5 times in 6 months, I enlisted the help of the station sergeant. I gave him a 1:2500 plan of the estate and, at my request, he went through the station’s day book for 6 months past, putting a red felt-tip dot against the address of each recorded burglary.

“He returned the plan to me saying “I’ve done what you asked and it looks like a bad case of measles, but I’m none the wiser.” As soon as I saw the plan I was immediately the wiser. The “measles” were overwhelmingly clustered around particular styles and types of dwellings, and the 3-storey walk-up open-plan flats, where the widow lived, were many times more likely to be burgled than (say) the semi-detached houses.

“I subsequently extracted £30,000-worth of additional fencing from my bosses to enhance security. (Quite a lot in 1974.)

“The point of the story is not that I was cleverer than the police sergeant; I’m sure I wasn’t. The point is that a policeman’s eyes see a residential area one way, and a housing manager sees it another. Who knows what might be achieved if lots of people could see the data and bring their distinctive perceptions and intelligences to its analysis and interpretation?

What indeed? Simon Dickson is a bit dubious about how easy it will be for government to do this; Steven Feldman (who I think we could fairly call a sceptic about Free Our Data – which is fine; an unopposed theory has no strength) has pointed out that postcodes sometimes give more detail away than you’d think (personally, I suspect that domestic violence will be excluded from these visible crime stats).

So we’ll wait to see. By Christmas? Sounds fun.

(Crossposted with the Technology Guardian blog)

Guardian praises Free Our Data (OK, well, not so surprising..)

Friday, July 18th, 2008

This morning’s Guardian has as its third “leader” (the opinion slot where the paper points to issues of the day), which is always “in praise of…”.

And today it’s In praise of “Free Our Data”. Hey, we’re chuffed.

The piece itself says (in part)

Businesses and others could use the data to map cheaply where crimes happen, or how much traffic is on the roads. Enthusiasts for cliff-climbing could share tidal forecasts. Those against argue that the Ordnance Survey’s work is not entirely paid for by taxpayers, or warn that it could lead to the privatisation of all data collection. These are serious points, and they should be taken into account. But the momentum is in favour of freeing up data; Cabinet Office minister Tom Watson boasts that he wakes up and immediately thinks “How can I free another dataset?” One hopes that is not literally true, but the sentiment is appreciated.

I don’t know, I like the idea of Tom Watson getting up having thought about a new dataset to make available. Heaven knows there are plenty of them.

But please go to the site and join in on the comments, which includes one with some interesting points about the British Library. (I’m not certain of the funding status of the BL, so don’t know if it would fall under the FOD umbrella or not.) Opinions? I lean towards the idea that the BL’s manuscripts are pre-existing data, and so there has to be some sort of cost involved in getting them into digital form…

A “fetishistic” attitude to privacy is holding back crime mapping, says Heather Brooke

Saturday, June 28th, 2008

You may have heard of Heather Brooke: she’s one of the campaigners who got MPs’ expenses and spending put into the public eye through the Freedom of Information Act and a great deal of determination.

Now she’s written a terrific article for The Times about crime mapping, and why it’s needed:

The police in Britain, however, feel they “own” crime data and the public have no right to know what is happening. Yet access to criminal incident data is vital, as it allows the public to judge the effectiveness of the police and crime policies. In a void of ignorance, a politician or police chief can claim anything he likes about crime: that binge drinking is endemic or under control, that muggings are increasing or falling, that policing is working or failing.

The police can also hide their failings. Northumbria police claimed that only three crimes of note had occurred one weekend in May, yet a freedom of information request revealed that, in fact, there were more than 1,000 incidents, 161 of them violent.

As she points out…

Shamefully, the Information Commissioner has objected to the plan of Boris Johnson, the Mayor of London, to allow people to know what crime happens in their street, arguing that it would breach the privacy of the victims of crime. But the Data Protection Act does not prohibit personal information being disclosed. Its purpose is to ensure that such disclosure is for a legitimate purpose.

Yet again a policy that would be of great public benefit is being blocked by an unthinking, fetishistic attitude towards privacy. A balance can easily be struck between the privacy of those reporting crimes and the overall safety of citizens. The only people made safer by the current policy of wilfully enforced ignorance are poorly performing police chiefs.

That “fetishistic” sums it up perfectly. Just as “health and safety” has become the refrain for anyone trying to stop someone else doing something in the physical world, so “privacy” has become something that is used to block anything that might disrupt the status quo.

But disrupting the status quo is what Brooke, and indeed the Free Our Data campaign, are all about. It’s difficult, changing peoples’ mindsets: but if you keep pointing out that things are wrong about how it’s being done now, eventually you set up enough cognitive dissonance that something does happen. Brooke has done a huge amount to that end with MPs’ expenses (even though some are now trying to squeeze around it). We find common cause on data like this. The struggle continues.

BBC’s iPM looks at crime mapping in Chicago

Tuesday, June 17th, 2008

The BBC’s iPM – the radio programme whose topics are suggested (though “chosen” would be overstating it) by its listeners – has looked at the topic of crime mapping, investigating how it’s done in Chicago.

There are two pieces: talking to Everyblock, the followup to chicagocrime,.org (which did the original crime mapping idea, building on the release by the Chicago police of the crime location data); and then talking to Jonathan Lewin, information services division commander in the Chicago police’s official crime mapping effort, and its GIS manager Joseph Kezon. You’ll have to visit the page to get the audio.

Notable points from the blog post:

Not everyone, of course, is happy with crime mapping. On the programme this weekend we’ll hear from groups who worry that crime-mapping could be counter-productive, affecting house prices, increasing fear of crime, and leading to areas being stigmatized.

Ah yes, the fear of knowing too much. Why don’t we just buy houses without ever seeing them? Why do we bother getting them surveyed? If house prices are affected, might it not also push up prices in places that don’t have crime? I’d have thought that more places would show low crime than will show high crime (because crime tends to cluster, for reasons that in hindsight are generally obvious), so that’s a net benefit for house prices; not that those should be the start or end of any conversation. (Please.)

Also amused by one of the comments:

I live in Surrey and we only have 10 police officers and all the police stations have been closed. They appear once a year for a major event and then we don’t see them again for 12 months. We do have a helicopter, loads of speed cameras, town centre cameras and a new video speed check van.

I think the police had a choice of more police on the beat or the helicopter and they decided that the helicopter looked like more fun. It hasn’t caught any criminals yet but they will produce statistics at the drop of a hat to assure you it is essential to crime busting.

The helicopter is very annoying and it would be useful to know the areas where it flies so potential house purchasers could avoid these areas. It flies around and around in pointless circles until residents are forced to report crimes in the next district so that it will fly away.

Don’t get me wrong, I don’t want more police on the beat. I want them to sell the helicopter and give me my money back.

Yup, helicopters are expensive to run. Maybe if there was a crime map you could decide where it should go more quickly..?

Is the Information Commissioner against crime mapping? Actually, no

Monday, June 2nd, 2008

Mike Cross has got a statement from the Information Commissioner’s Office (ICO) on the proposed crime mapping by the new London mayor. Here it is in full:

“The ICO recognises the benefits of crime mapping to help police forces identify crime patterns, for instance to help plan the deployment of officers. However, when considering making such information public, it makes sense to think through what the intended benefits are and to weigh these against the potential privacy intrusion involved. We believe that this will involve carefully considering what level of detail, both of the offences themselves and where they took place, should be placed in the public domain. There is a risk that revealing exactly where a particular crime took place could lead to the identification of victims who may wish to remain anonymous. Even in circumstances where an individual cannot be confidently identified, there may be a risk that releasing information about a crime may lead some to infer the identity of the victim. It may also cause concern to those living in the locality, especially those with particular concerns about their security, for instance those living in some form of refuge. There is a need to balance the intended benefits of releasing such information against the potential intrusion or concern this might cause individuals.”

Points to note: the ICO does not come out against it, explicitly. It says that there is a need to balance benefits and risks. To be honest, it is vague to the point of pointlessness.

As to “revealing exactly where a particular crime took place”, that is done all the time by police reports (and then by media reports). And the proposals put forward by Boris Johnson (at least in his manifesto) are specific about *not* giving precise locations of crimes – it was clearly recognised. To reiterate, quoting the manifesto,

We believe there should be three kinds of map for each area for three categories of offences:

(a) where privacy is less of an issue: at their exact location with a pinpoint showing exactly where the offence occurred;
(b) where privacy might be an issue: identifying a 300 metre long street section within which an offence occurred; and
(c) where privacy is an issue: identifying the nearest whole street within which an offence occurred.

(Assaults would be in category (a); sexual attacks and domestic violence in (c), for example.)

It seems likely the ICO hasn’t looked at this, even though it’s been in the Tory manifesto, and of course here on our blog (aren’t we required reading for the ICO?).

So now it’s up to the police and, of course, to the mayor’s office. The clock is still ticking…

The police and crime mapping: and another perspective from the Observer

Sunday, June 1st, 2008

The Observer newspaper has an article today by Nick Cohen – a truly criminal approach to policing – which looks at the broader question of policing, and how setting targets can have a negative effect on other aims. It seemed worth quoting in the context of the earlier discussion about why the Met police appear to be against crime mapping (even if other regions have dipped their toes in).

Cohen writes:

Last week, Harriet Sergeant of Civitas described a police service which was close to incapable of doing its job. In a think-tank pamphlet, she delivered a devastating condemnation of an enclosed and self-referential bureaucracy which operated without regard to the wishes of the people who paid for it.

We now spend proportionately more than any other developed country on policing, she pointed out. The Home Office used targets to run it and delivered funding and bonuses to chief constables who filled its ‘sanction detention’ arrest quotas.

The first perverse consequence was that although the public expected the police to keep the peace, an officer who successfully stopped trouble was not rewarded because no trouble meant no arrests. More seriously, the police played the Home Office game by going for trivial offenders rather than serious criminals. Solving the case of a child who steals a Mars bar earned as many points as solving a murder. It made more sense to arrest rowdy children for ‘harassing a tree’ than to begin the hard work of tackling a potentially homicidal teenage gang.

Chris Dillow, author of New Labour and the Folly of Managerialism, describes Brown’s Mullettry as a marriage between Old Labour’s Fabian belief in the centralised state and Thatcherites’ worship of management consultants. Between them, they have spawned a bureaucracy which despises democratic accountability and, worse, does not and cannot work.

(“Mullettry” isn’t a reference to hairstyles, but to the senior officer to the fictional Inspector Frost.)

The whole article’s worth reading and considering while asking the question: would this police force want to release the data on crime mapping?

In today’s Guardian: how Inspire was watered down

Thursday, May 22nd, 2008

German floodInspire sounded like it would be such a great idea: data would be swapped for free between governments, everyone would benefit.

But practice and theory tend not to be next-door neighbours. And so it has proved with Inspire, which as this week’s Guardian story – An Inspired debate about access – points out, hasn’t lived up to early hopes.

First, some very good news. Civil servants revealed last week that the British government has begun work on a system to make all the geospatial data it holds on the natural environment available for free inspection and re-use. Now the bad news. In this context, “free” means we will still have to pay to download much key data, especially if it is to be published or otherwise used commercially.

The British government has until May 2009 to “transpose” Inspire’s measures to UK legislation: the Department for Environment, Food and Rural Affairs (Defra) is leading the work. In a culture where public agencies jealously guard the intellectual property in their databases, and often have to recover the costs of running them, the new regime may face some opposition.

“It is clear that Inspire will require organisations to collaborate better,” David Lee, one of the Inspire project team at Defra, said at the Public Sector conference in London last week. He revealed that the department is considering creating a geoportal, which would allow anyone to track down any geospatial data set, regardless of who owns or maintains it. “The vast majority of transactions to members of the public will be free,” Lee said.

However it is not quite as simple as that. While the Inspire directive requires “metadata” (machine-readable data about data) to be provided free of charge, access to the databases themselves may be charged, especially if it is for commercial use.

And so that dichotomy continues: commercial use versus “private” use. Everyone, it seems, should have Chinese walls in their minds – how else can you be sure that knowledge gained in one’s personal life won’t be exploited in their commercial existence?

Land Registry surcharge could fund free OS data surprisingly cheaply

Thursday, March 27th, 2008

Sold signs outside houseOne suggestion that has been made by Robert Barr (of Manchester Geomatics) and echoed recently by Ed Parsons on his blog (though I think Ed came up with it independently) is that Ordnance Survey’s non-refined data (that is, the stuff it does as part of its public task, which the Cambridge economics study of trading funds interpreted to be its MasterMap and Large Scale Topo) could be made available for free by making up any funding shortfall from a surcharge on Land Registry transactions.

The reasoning: most LR transactions involve OS mapping.

According to the study, that would cost between £12m and £30m in foregone revenue.

So how much would you have to add to Land Registry transactions to make up that amount? It sounds like an awful lot of money to generate.

Here are the figures I’ve culled from the Land Registry’s performance data for the past three years on the number of transactions.

Number of registrations 2004/5 2005/6 2006/7 Mean 04-06 As % of total
first registrations 297,405 309,609 304,391 303,802 4.3
discharges 2,486,875 2,502,318 2,605,620 2,531,604 35.8
mortgages 2,680,128 2,627,999 2,723,530 2,677,219 37.9
transfers for value 1,378,200 1,270,867 1,480,819 1,376,629 19.5
leases 167,234 173,610 197,546 179,463 2.5
Total 7,009,842 6,884,403 7,311,906 7,068,717 100
Total w/o discharges 4,522,967 4,382,085 4,706,286 4,537,113 64.2

With millions of transactions, it looks like raising £12m – £30m wouldn’t actually be too hard. “Discharges” are the ending of a claim to a legal title – generally, though not always, the end of a mortgage. They attract no fee at present. Other LR charges range from £2 (for a search) to £700 (for first non-voluntary registration of a pricey parcel of land). Most of the charges, though, are £20 – £40 and upwards.

So to find the £12m that the trading funds report suggests OS would lose solely from non-discharge transactions would mean adding £2.65 to the cost of each LR transaction.

If we take the loss in revenue to OS as £30m, then it means adding £6.61 to each transaction. It’s not more than the cost of any transaction (except searches – which aren’t the same as the “searches” one does when buying a house; those go through your local authority), and compared to the cost of the typical transaction – say, the average £180,000 house purchase – it’s peanuts.

Right – that’s the analysis done. Now we just need to find a minister who is in charge of Land Registry and Ordnance Survey and can tweak the legislation (it doesn’t need primary legislation, surely?) to make these changes. And we’re done.

This analysis also appears (without the fun table) in today’s Guardian: Land Registry holds key to free OS.

In the Guardian: ‘we’re now in sight of victory’; so what next?

Thursday, March 20th, 2008

Following on from the trading standards report, today’s Guardian examines what it could mean, and what the government – and other – response so far has been.

In sight of victory notes about the study that:

The findings will be hard to dismiss. Unlike previous studies, they are based on hard figures from the trading funds affected. It also takes a holistic view, for example taking into account the overall cost to society of the extra taxation needed to pay for free data.

And then there’s the reaction:

Britain’s leading expert in public sector information policy, Professor Richard Susskind, chair of the Advisory Panel on Public Sector Information, described the report as “precisely the kind of detailed, systematic and rigorous economic analysis of trading funds and PSI re-use that APPSI has been recommending since 2003. We hope this represents the beginning of a new era of open and sophisticated thinking about the economics of PSI.”

Michael Nicholson, chair of the industry association Locus, said the study “breaks new ground. It is the first time the consequences of exploiting public sector data have been reviewed from a socio-economic perspective in this way”.

Ordnance Survey, the agency that would be most affected by a change in policy – and whose raw data the report says would benefit the UK most – said the study is “an important input to the wider debate”.

Ed Parsons, the OS’s former chief technology officer, noted that “the ludicrous merry-go-round of government departments paying [each other] would disappear, reducing costs and increasing the use of geographic information (GI) within government … the relatively small GI industry in the UK would flourish”.

(We should also note that Ed is not in favour of an OS that is totally directly funded. In the same blog post he says that

OS in the position where its continued operation and the quality of its data is reliant on a subsidy from government [would be] a disastrous position which could result in a USGS like reduction in funding if political priorities change.

We take that point on board – and we’ll deal with that in coming weeks.)

So what does government, which now has stacks of data about free data’s benefits, say?

Central government was less enthusiastic in its responses. The department for Business, Enterprise and Regulatory Reform, which oversees the running of trading funds, implied pricing policy would continue as before. While the government would set out clearly what information it needs for its public task, to ensure that such information “is made available as widely as possible for use in actual and potential downstream markets” the principle of recovering costs will remain in the next spending review (after 2011).

Yes, you can groan now.

“The underlying principle will be that information collected for public purposes will be made available at a price that balances the need for access while ensuring customers pay a fair contribution to the cost of collecting this information.”

In other words, DBERR knows much better than economists using data gleaned from the organisations involved what’s right for the economy. It’s a bit shocking, really.

Over to you: what do you think should be the next steps in the campaign? Our first thought is to examine how effective a system where Land Registry registrations part-fund Ordnance Survey, since registrations are (in economic lingo) inelastic. Anyone know where the data about the number of registrations via LR for the past 10 years could be found?