Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens


Postcodes to be free? But which ones?

December 9th, 2009

The BBC has a piece saying that “postcodes” will be free from 2010:

Currently organisations that want access to datasets that tie postcodes to physical locations cannot do so without incurring a charge.

Following a brief consultation, the postcode information is set to be freed in April 2010.

….

The dataset that is likely to be freed is that which ties postcodes to geographic locations. Many more commercial organisations use the Postcode Address File (PAF) that ties post codes to addresses. Currently access to either data set incurs a charge.

In October 2009 the Royal Mail took legal action that cut off the access many websites had to PAF data.

(You might remember that one.)(

Sites that used the postcode feed included Job Centre Pro Plus, HealthWare (locates nearby pharmacies and hospitals), Planning alerts.com (monitors planning applications), Straight Choice (finds out who sent political leaflets).

That’s quickly contradicted however by the email that came around from the Royal Mail, noted by Steve Feldman, coming from Giles Finnemore, Head of Marketing at the Address Management Unit of the Royal Mail:

You may be aware of a story on the BBC website today that Government is planning to give anyone free access to postcode data.

Access to postcodes is already, and will continue to be, free to every citizen via www.royalmail.com/postcodes4free.

(Which is a nonsense. It’s true, but it’s also nonsensical, because the postcodes4free page requires registration and will only give you a limited number of postcode lookups in a 24-hour period. Which, if you think about it, is absurd: why does the Royal Mail want to make it difficult to address letters? You need to have an address list if you want to generate postcodes; if you didn’t have the postcodes, where did you get the addresses?)

For the avoidance of doubt PAF(r), the Postcode Address File, remains the intellectual property of Royal Mail and is supplied and used under licence. The new and recently published licences come into effect from April 2010. There are no plans for that to change.

Maintaining a world class postal address file requires significant ongoing investment and it is right that organisations who obtain value from using the file pay to do so.

We are aware of no plans for Government to pay Royal Mail for businesses and organisations to use our address file.

And it’s also contradicted by the Royal Mail’s press release page, which at present (December 9) has nothing about postcodes.

However, it may well be that the PostZon file – or more precisely, the long/lat lookups for every postcode – will be available for free next April.

Daily Telegraph: making stuff free can create revenues

December 9th, 2009

Hey, look, even the Daily Telegraph – hardly a home of the idea of the free lunch – is reprinting Breaking Views pieces which point out that making data free brings bigger benefits.

UK map giveaway throws bread upon the waters pretty much sings from our songbook:

The Met Office and the Ordnance Survey are unlikely candidates to stimulate another revolution. The weather forecasts may be accurate (sometimes) and the maps beautiful, but as businesses, neither is going anywhere. This is no surprise, since neither is really suited to becoming a proper commercial enterprise.

Yet the data they own is, literally, invaluable. Made freely available, all sorts of would-be entrepreneurs could exploit it to build businesses beyond the dreams of the public sector. The slightly geeky approach needed to be a successful internet entrepreneur is commonplace among mapaholics and weather nuts. Given the raw material, they could make a thousand businesses bloom.

The proposal unveiled this week is vague – a consultation document is promised later this month. The ability of the civil servants to emasculate any good idea should never be underestimated. But this is one whose time has come.

Given their tiny profits, selling off the Ordnance Survey and Met Office would raise minimal amounts. Giving away the data will undermine profits, but the benefits in terms of corporate taxes should be much larger.

Thanks. We knew you couldn’t keep a good idea down.

Outrageous. Incredible. International expert was spoken word only even within OS

November 27th, 2009

Let’s just remind ourselves what it was that Sir Rob Margetts, chair of Ordnance Survey, said at the launch of OS’s proposed new strategy (which is now in little pieces all over the floor since Gordon Brown and Tim Berners-Lee announced the end of derived data and the freeing up of mid-scale mapping, but anyway) back in April:

“We came to conclusion that the cost to government in the first five years [of a free data model] would be between £500m and £1 billion. That wasn’t the only reason that we discarded it. We did, with outside help, a review of equivalent organisations around the world.“

Who, I then asked, was the “outside help”? OS responded:

With regard to the International Comparison of Geographical Information Trading Models Study, outside help was provided by senior officials of those Institutions contacted.

In the case of the United States of America, as senior officials of the United States Geological Survey (USGS) were unavailable, Mr. David Cowen, Distinguished Professor Emeritus at the University of South Carolina, kindly provided us with an in-depth overview of the state of public sector GI data in the United States, including USGS. Mr Cowan is a former chair of the Mapping Science Committee of the United States National Research Council and is chair of the National Research Council’s Committee for the study of Land Parcel Databases.

The document was also reviewed by an internationally recognised expert in Geographical Information and National Mapping who agreed with the analysis and conclusions.

This latter bit intrigued us. An “internationally recognised expert”, eh? Except it turned out that he or she did not want to be identified, although he or she works or has worked full-time for a foreign mapping agency, and read the study for free. And that OS transacted everything with the expert by spoken word:

A copy of the report was provided to the person concerned and engagement on this matter was conducted orally with no permanent record made of these conversations.

And now in response to my latest Freedom of Information request for

copies of all emails and/or documents internally relating to the decision to choose this person – for example, discussion of who would be suitable candidates or who would not be suitable candidates to carry out the review of the report

OS replies:

There was no decision process in place to find suitable candidates. An opportunity presented itself to request the opinion of a global expert in this field which was undertaken orally. The resultant opinion was expressed orally and there was no permanent record made of these conversations.

So here’s what happens. You have a report. You happen to bump into an old mate. “Hey, want to read my report?” you say. “Sure,” they say. They read it. “Seems OK,” they say. You go back to your office and tell people “I met X who says it’s fine.” Even though the report is a thrown-together farrago of disconnected information about various national mapping agencies and their charging methods, combined with an unrelated chunk of poorly displayed data about national GDP versus national R&D expenditure, which cannot by any reasonable measure be claimed to justify anything about any charging model.

This then becomes “The document was also reviewed by an internationally recognised expert in Geographical Information and National Mapping who agreed with the analysis and conclusions.”

If there is anyone at Ordnance Survey who is prepared to defend this course of events, could they please get in touch? Or even the international expert, who is very welcome to comment anonymously to explain whether they think OS’s representation of their opinion is justified. Comments are open.

Tim Berners-Lee and Nigel Shadbolt on the benefits of open data

November 18th, 2009

TBL and Nigel Shadbolt, who are together pushing along the open data idea in government, have an article in the Times (London) of November 18 about the benefits of free data, following on from the announcement yesterday:

Data has a particular value in that you can combine it with other data to discover new things. When in 1854 John Snow took the deaths from a cholera outbreak in London and plotted them on a map, he was able to illustrate the connection between the quality of the source of water and cholera — the world changed. In March the Department for Transport released three years’ worth of data about the location of accidents involving cyclists. Within 24 hours someone had converted this data to create cycle-accident route planners that avoid the black spots.

Government data is a valuable resource that we have already paid for. We are not talking about personal data but data that tells us, for example, about the amount and type of traffic on our roads, where the accidents are, how much is spent on areas where these accidents occur. This is data that has already been collected and paid for by the taxpayer, and the internet allows it to be distributed much more cheaply than before. Governments can unlock its value by simply letting people use it. This is beginning to happen in a number of countries, notably in the US under the Obama Administration, and in June Gordon Brown asked us to advise the Government on how to make rapid progress here.

(The fun thing here being that OS would argue that its data has not been collected and paid for by the taxpayer because it’s a trading fund. Unfortunately this doesn’t hold up in front of the point that (a) almost all of its data was collected while it was not a trading fund (b) half its revenues do come from the taxpayer, in the form of licences from public organisations.)

As all of this data becomes available, we have to look for the joins between it. A new set of standards for the web is emerging that allows us to link data from different sources. Everyone knows that web pages have addresses that identify them, allowing you to navigate around and find what you want. To make the web of linked open data work we also need to give identifying addresses to the objects and properties that make up the basic information in pages, spreadsheets or databases.

Think about the practical applications. If Companies House referred to companies using these new open, uniform identifiers, then other people who needed to talk about companies could use these whenever they referred to a company. If all websites that make data available about companies point to the same identifier for a company, then it’s possible to pull that data together very easily — whether its data about stock price, a product or a director. This is one of the core principles at the heart of the web of linked data.

None of this works unless the data is there in the first place. But when it is, innovation flourishes. Maybe someone uses the web to show schools close to you and their Ofsted reports, or the planning applications that might affect you, or the allotments available to use, or the crime rates in your area. Data is beginning to drive the Government’s websites. But without a consistent policy to make it available to others, without the use of open standards and unrestrictive licences for reuse, information stays compartmentalised and its full value is lost.

So there you have it: the free data concept is right there at the heart of government, with extra semantic web power from the person who invented it. That’s good. That’s very good.

Gordon Brown announces OS maps to be free online

November 18th, 2009

Do we scent victory? Hell yeah. It seems that the prime minister plus the chief secretary of the Treasury plus the inventor of the world wide web collectively outrank Vanessa Lawrence, and so Gordon Brown was able to declare at a seminar at No.10 yesterday (to which I was invited, thanks for asking) that

Today, some of you may know, we are opening up Ordnance Survey information – one of the first recommendations that Tim Berners‑Lee made to us with Nigel Shadbolt in the work that they are doing. We are making Ordnance Survey material available to the up to a certain level in a way that it was not available free of charge before.

The Guardian has the story: Ordnance Survey maps to go free online:

The government is to explore ways of making all Ordnance Survey maps freely available online from April, in a victory for the Guardian’s three-year Free Our Data campaign. The move will bring the UK into line with the free publication of maps that exists in the US.

Gordon Brown announced the change at a joint event in London today with Sir Tim Berners-Lee, the inventor of the World Wide Web, who is now information tsar advising on the handing over of private government data to the public.

The government has been inspired by the success of crime mapping where “data openness” is helping citizens assess the safety of geographical areas.

Today’s announcement will be followed by a speech, due next week by the chief secretary to the Treasury, Liam Byrne, explaining how the freeing up of data, alongside the scaling back of other functions of central government, could lead to a “smarter state”.

Our understanding is that Liam Byrne was key in getting this pushed throughL since it involves financial risk – OS won’t be getting that income – the Treasury has to approve it.

Key points: it involves “mid-scale” maps from 1:10,000 upwards; and it kills off the “derived data” rows that government departments and everyone else has been having for so long. Derived data will have a stake through its heart.

Oh, and – this “free” will extend to being free for commercial use. That’s right, you’ll be able to build a business with it. Though it’s not clear yet whether you’d be able to take the maps and create *printed* ones. Must ask about that.

There will be a consultation starting in December. We’d urge any customer of OS to add in their views. And we’d urge any would-be customer who would otherwise not use the data to add their views too.

Quite where this leaves OS’s “hybrid strategy” isn’t clear. And OS doesn’t seem very clear about it either. Vanessa Lawrence wasn’t at No.10, and nor was anyone from OS, which seems surprising – you’d think they’d want to bask in the reflected glory of being praised by Tim Berners-Lee for the quality and usefulness of their data, surely?

When we asked this morning how much foregone revenue this means (since obviously giving away maps you used to charge for means less income), OS said it was “not in a position to make any comment at this time”, which seems surprising, again, because you’d think that it would have given Liam Byrne very clear indications of how far the roof would fall in if it were to do this.

Text of Norwegian mapping service announcement, in English

November 14th, 2009

Here’s the best shot that Google Translate has come up with in translating the web page at the Norwegian mapping service announcing its forthcoming free map service. There are some important extra points which might be worth noting – such as that it’s going to be for an experimental period of six months in the first instance. (And you’ll have to allow for the fact that it’s not quite been translated into English.)

Free map services [Published: 12.11.09, Updated: 12.11.09]

Now everyone can access the latest maps around the clock. Together with our partners we are releasing through the Norwegian Mapping Authority a number of their map services – by Mapping Manager Anne Cathrine Frøstrup

From 1 December, you can click on the state map sites and get direct access to digital map services – absolutely free.

“We have the best and freshest maps foundation in Norway, and now we share this with everyone. This shows that cartography is visibly for the benefit of society, and is entirely in line with government policy of more openness,” said Hydrographic chief Anne Cathrine Frøstrup.

Map Services released, covering both sea and land, and regularly updated. They should meet the needs of the vast majority of us.

Hoping for creativity: It is free to develop their own functions related to the maps, and in the long run, this can provide a range of new opportunities.

“Be creative and use our maps to develop new exciting additional services,” said Frøstrup.

With the release of the maps she hopes that creativity will flourish. Inventive souls can tie together maps and other map information in new and clever ways.

Availability cookbook: with some computer knowledge, it is possible to use the new map services. On the website of the Norwegian Mapping Authority, you will find a basic cookbook that gives you the recipe for how to integrate the various solutions.

The links to the services will operate from 1 December.

In the first instance we are talking about a test period of six months. The scheme will be constantly evaluated and improved. Works and services so we have planned, will the offer be made permanent. [Not sure how this last sentence should be interpreted.]

Norwegian mapping authority frees up maps

November 14th, 2009

Oh, those crazy Norwegians. They’re giving away maps for free. Within limits – it’s limited to “individuals and non-profit associations”. So that’s half a step towards free data.

Interesting that this is also including maritime information – the stuff that the UK Hydrographic Office sort-of provides; it’s rumoured to be on the list for privatisation, however.

And of course Norway was one of the countries in the OS’s woeful International Comparison report. Apparently it gets an unknown amount of government grant, but operates a “full cost recovery” system (6.1.4 in the report).

Arguably this is similar to the OS’s OpenSpace project, which is free-ish availability of data for individuals and non-profits… as long as they don’t get too big. What’s not clear in the Norwegian example is what its rules on “derived data” are. That would be interesting to know from any Norwegians.

From this Google translation page of the Associated Press story in Norwegian:

(AP) Soon, you can use much of the information at the Norwegian Mapping Authority on your private website, and take up battle with Google Maps.

From 1 December this year, you can retrieve detailed information from the Norwegian Mapping Authority, and bake it into your own web pages. Totally free, if you are an individual or operate a non-profit association.

It follows the Norwegian Mapping Authority in the footsteps of Google and the Norwegian Meteorological Institute, who shares the information it holds on to the free public use.

There is talk of detailed maps and geographic information that the Norwegian Mapping Authority has been sitting for a long time, and that until now only been available for those willing to pay.

“The data has been available through the programs that we manage, but now it will therefore be direct access to the service,” explains Erland Røed the Norwegian Mapping Authority to VG Nett.

Like for example, Google’s maps will be extended the opportunity to mix together information from several sources, and then add this on top of the map from the Norwegian Mapping Authority.

You should read the whole page (so I’m not going to copy it all here). But it does sound like the Norwegian Authority is getting with the program in a big way.

….At the State Map Verks web pages, users can decide how utstnittet of the map should look like, the layers of information to be included and finally get a clip of code.

…The system located at the bottom of the State Map Verks solutions [is] also at the forefront of the development of open software.

“Operating systems and databases we use are free software, so this is done by the book. It’s gone out 80 million map images of this year, says Røed. “The public and private individuals who have gone on our site has generated a lot of use before it is released freely, “he says.

By releasing data free on the way the Norwegian Mapping Authority do hope that the users themselves to come up with good solutions. A few examples of how users can generate more information from the maps of the Norwegian Mapping Authority is sports.

See chart to the Norwegian Mapping Authority, for example, be used as background during a real-time tracking of Færderseilasen with all available information as a true marine gear.

Such tracking is technically possible even with the Google service Google Maps, but without details that a real chart can offer.At the State Map Verks map is the depth measurements, lighthouses, beacons and all the relevant information for sailors, where Google Maps only shows the blue sea.

Similarly shows the topographical maps of the Norwegian Mapping Authority highly detailed rendering of the terrain, long more accurate than Google Maps.

There is also a reader discussion of site boundaries, which has a translation too.

International expert fun rumbles on

November 10th, 2009

Tom Watson, the former Cabinet Office minister, has also weighed in to the International Expert fun. He filed an FOI request asking for

any briefing papers, emails or any other documents relating to Charles Arthur of the Guardian and the media interest as to the identity of the “internationally recognised expert”.

I am particularly interested in the discussion that may have taken place in regard to his enquiry about the status of the “internationally recognised expert.”

Where possible, I would like to see the advice that was given to civil servants, ministers and the shareholder executive about this matter.

OS’s reply (in a TIFF – a giant uncompressed image format that can crash many machines if you don’t have enough memory installed; what’s wrong with output to PDF if you want to ensure it’s in the format you created it?):

Correspondence with and about Charles Arthur’s enquiries of Ordnance Survey on the “internationally recognised expert” involved only Ordnance Survey staff and Mr Arthur. No briefings, emails or any other documents were communicated to other civil servants, the Shareholder Executive or Ministers on this matter.

I’ve now filed another FOI request seeking to know what internal communication there was about the international expert:

I request copies of all emails and/or documents internally relating to the decision to choose this person – for example, discussion of who would be suitable candidates or who would not be suitable candidates to carry out the review of the report.

I agree that, for reasons of privacy, some names of those considered and of some of the senders/receivers of the information internally may have to be redacted. I would expect this to be kept to a minimum.

However as the study is officially the responsibility of OS’s chief executive I would expect that where the chief executive’s name appears in such documents that it should not be redacted as the chief executive was in overall charge of the decision. Similarly I would not expect the names of any OS board members to be redacted in such correspondence as they must have a secondary responsibility for the report.

Let’s see how that goes. The answer is due by 2 December.

PDFs are bad for open government, says Sunlight Foundation in US

November 7th, 2009

This is always worth remembering:

Government releasing data in PDF tends to be catastrophic for Open Government advocates, journalists and our readers because of the amount of overhead it takes to get data out of it. When a government agency publishes its data and documents as PDFs, it makes us Open Government advocates and developers cringe, tear our hair out, and swear a little (just a little). Most earmark requests by members of congress are published as PDF files of scanned letters, leading the Sunlight Foundation and others to write custom parsers for each letter.

I know that a lot of the efforts going on in the data.gov.uk channels are about finding effective ways of parsing data. The hope has to be though that very little of that involves finding ways of reversing data that has been output to PDF. The point being of course that turning PDF into useful data is, in the famous quote, “about as easy as turning hamburger into cow”.

Back to the Sunlight Foundation again:

Here at Sunlight we want the government to STOP publishing bills, and data in PDFs and Flash and start publish them in open, machine readable formats like XML and XSLT. What’s most frustrating is, Government seems to transform documents that are in XML into PDF to release them to the public, thinking that that’s a good thing for citizens. Government: We can turn XML into PDFs. We can’t turn PDFs into XML.

And another word for Flash. Ah, Flash:

Flash isn’t off the hook either. Government has spent lots of time and money developing flash tools to allow citizens to view charts and graphs online, and while we’re happy the government is interested in allowing citizens to do this, Government’s primary method of disclosure should not be these visualizations, but rather publishing the APIs and datasets that allow citizens to make their own

The comments are worth it too, such as Adrian Holovaty: “If I had a dollar for each hour I’ve spent trying to finagle raw data out of PDFs, I could afford Adobe Photoshop.”

And the rather scary one from Michael Friis: “Here in Denmark Parliament publishes many ancillary documents as PNGs.” Which is quite scary, though in line with Ordnance Survey’s tendency to release FOI requests as TIFFs.

Digital engagement, widening and public data getting analysed… in private

October 30th, 2009

Stephen Timms reports that there’s been good progress in Making Public Data Public.

As the Digital Engagement blog notes:

So far our request for developers to “get excited and make things” has so far exceeded our initial expectations. Not only is the number of people signing up to the developer forum higher (currently more than 1,300), but also the discussion board is very active with a healthy list of ideas for the site and, perhaps most excitingly, a few applications are beginning to see the light of day.

And also:

Working in partnership with Guardian Professional, we held 3 developer days hosted at The Guardian’s Kings Place offices in central London on the 14th-16th September. As an organisation they were best placed to help us undertake this task, having built a community of talented developers and opened up their API. You can have a look here at the excellent postcode paper concept and the rather wonderful traffic data visualisations here, which were just two of the many ideas for applications that emerged over the course of the camp. Ideas about their priorities for further data releases (to add to the 1,100 datasets currently on the site) were shared and important foundations for further iterations of the HMG Data site were laid.

There’s a certain irony in the fact that the sessions at the Guardian were held under such secrecy that I didn’t find out about them until the week after. More posts on that later…

Tim Berners-Lee to help UK government build single data access point

October 29th, 2009

Computer Weekly reports that Tim Berners-Lee has been asked by the government to develop a single point of access for public data – as Stephen Timms, who has taken over where Tom Watson left off in the Cabinet Office, reports progress in “making public data public” (a concept that, when you think about it, seems a bit strange – as in “shouldn’t that have been done from the outset?”).

According to Computer Weekly, Timms told an RSA/Intellect event that

information is the “essential raw material” of a new digital society. “Government must play its part by setting a framework for new approaches to using data and ‘mashing’ data from different sources to provide new services which enhance our lives. In particular, we want government information to be accessible and useful for the widest possible spectrum of people.”

Well, minister, if that’s truly what you want, then you’ll make it free of charge, and free of copyright restrictions. It’s as simple as that. Could we suggest something like Creative Commons? The US government seems to find it amenable. .

Timms said, “We are supporting Sir Tim in a major new project, aiming for a single online point of contact for government data, and to extend access to data from the wider public sector. We want this project for ‘Making Public Data Public’ to put UK businesses and other organisations at the forefront of the new semantic web, and to be a platform for developing new technologies and new services.”

Fine words. We’d like some actions to go with them. We’re hearing plenty of sticks being wielded over how people use the net – Lord Mandelson’s threats to file-sharers, for example – but the carrots for companies to build on something that really would benefit Britain, by using British data, seems to be stuck on a really slow train.

Part of the problem, of course, is that it’s almost impossible to put a figure on how opportunity cost is lost through the lack of access to this data – whereas the music industry can much more easily point to figures it’s produced (though you may argue about their provenance) to suggest precisely how much harm it’s suffering through untrammelled downloading.

Interesting to contrast, though, that when we asked the Royal Mail to specify precisely how much harm it was suffering through the use by ernestmarples.com of the postcode to lat/long conversion, it robustly declined to say.

Of course there is the Cambridge trading funds report, with its analysis of the opportunity cost of the trading funds regime. But this goes much wider – the Cambridge analysis didn’t look at the Royal Mail and postcodes, for example, which have become embedded into many systems’ location processing.

Computer Weekly again:

So far, 1,300 people have signed up to the developer forum and contributed to the discussion board on what the data could be used for. The Cabinet Office also held a developers’ camp where ideas were shared.

We’ll have more about the devcamp in a future post.

Deleting users, for security

October 22nd, 2009

As this site has been the target of some hackers (who were being paid by a Canadian pharmaceutical seller), I’m having to delete user registration here.

Sorry about that. Being hacked was a byproduct of having user registration, so one or the other has to go…

On the plus side, you do now get spiffy icons that will indicate whether you’re really the same person as when you last commented. So, not such a loss.

My thanks to Stefan Pause for his help and advice on securing the site.

And on the plus side, I’ve taken his advice and updated the permalink structure, which means you now get meaningful (if long) URLs. Win-win, I think. Plus I’ve got some fascinating hacker scripts to deconstruct.

Kent County Council wants you to recycle its data

October 20th, 2009


Have a look at http://picandmix.org.uk/:

Pic and Mix aims to increase public access to Kent-related datasets including those generated by Kent County Council (KCC). For the purposes of the pilot, we have brought together a sample of the most useful information. Where possible, it’s been provided in a format that allows it to be ‘mashed’ and customised. Please help us shape this initiative by suggesting additional data and ways in which we can improve this site. And if you do anything clever with the data, we’d like you to share that with us too!

The About page has more:

Last year, Kent County Council won Innovate08! Our idea had three elements:

  • To make publicly available information – things like crime statistics, employment information, business information – more
    accessible.
  • We also wanted to provide tools that would enable people to ‘pic and mix’ data to create customised information.
  • And last but not least, we wanted to provide a platform where people could share this information and discuss ways in which it could be used.

Winning Innovate08 meant we were given funding for a pilot project to see how people in Kent would respond to a resource of this kind. Our pilot project was intially launched with 25 small Kent-based businesses. With this new site we  hope to get the wider community involved.

So, how could Pic and Mix benefit you? Well, there’s a lot of information out there in a lot of different places. Rather than spend ages tracking down the information you need, we want you to come to a single place – picandmix.org.uk. For example, you may be looking for a care home for an elderly relative. You might want to mix this information with GP locations and bus routes. By plotting this information on a map you will be able to see which care homes are close to a GP surgery, and the bus routes. Another example might be a security company deciding where to focus its marketing efforts. They may want to mix office premises with crime statistics and use the information to plan a campaign.

Fascinating. We await developments – and news of same

Costing ernestmarples (and free data) vs paid-for

October 11th, 2009

Somewhat late, but better than..

In the Guardian on Thursday, we have the cost-benefit analysis – if somewhat cursory – of having Royal Mail charge for its PostZon database (as used by ernestmarples, though indirectly) and having it available for free. So, for example, did RM lose out through ernestmarples? Or did we taxpayers benefit?

In Who would really benefit if postcode data were free, we add it up.

Royal Mail claimed that Richard Pope and Harry Metcalfe, the duo behind the site, had caused it “loss”. As the PostZon database being accessed via ernestmarples.com – named after the man who introduced postcodes to the UK – costs about £4,000 a year to license, could it be right?

Some simple calculations show that in fact everyone else, including the government that owns Royal Mail, and perhaps even Royal Mail itself, would benefit from the data being free.

Pope and Metcalfe point out that ernestmarples.com, which queried other websites that provide PostZon data for its postcode-location conversions, fed a number of their other websites – including Job Centre Pro Plus (which used a postcode lookup to find jobs near you), Planning Alerts (which alerts you to new planning applications in your area) and The Straight Choice (used to file election leaflets by area).

Job Centre Pro Plus had 437,354 searches for jobs since March this year, according to Metcalfe. If only 0.001% of those led to someone finding employment and saved £100 in benefit payments, then ernestmarples.com has, overall, saved the government money.

And Pope points out that professional property developers used PlanningAlerts “since it allows them to look for opportunities/competition”.

If that led them to work worth more than £20,000, the 25% corporate tax rate means the government has received more in tax revenue than it has lost from Pope and Metcalfe’s non-licensing of PostZon. Pope also notes that “few councils were using the PlanningAlerts API [programming interface] since it was easier and cheaper than paying external consultants to hack they achingly bad internal systems.” He points to Lincoln City Council, where PlanningAlerts was used to generate the RSS feed and map for planning. Would it cost more than £4,000 for Lincoln to build a system to do the job PlanningAlerts enabled?

Furthermore, “I was told by someone at the Electoral Commission that they used the Straight Choice during the Euro elections to monitor parties,” Pope said. “The alternative would be paying for hundreds of field agents (which they can’t afford).”

Rufus Pollock, a Cambridge economist who co-wrote a study for the government on the economic benefit of making trading funds’ data free, calculates that making PostZon free would bring an economic benefit 50% greater than Royal Mail’s present revenues.

Subequently it’s been suggested to me that the cost of licensing is more like £1,200 rather than £4,000 – which makes the case for benefit from free data even greater.

Royal Mail threat likely to close ernestmarples.org

October 5th, 2009

The Royal Mail has sent cease-and-desist orders (via its lawyers) to Richard Pope and Harry Metcalfe, the web developers behind ernestmarples.org (which we’ve referred to before).

Basically, they’re saying that ernestmarples is accessing the RM’s postcodes-to-coordinates database without permission, and that their clients are suffering loss as a result, and so they should stop.

Read it all (including the letter) at http://ernestmarples.com/blog/?p=3.

Two immediate questions:
1) how can it be unauthorised to access a database that is publicly accessible through others? Though of course there may be lots of fine print in white on a white background in which you “agree” not to reuse the information for anything actually useful anywhere.

2) precisely how much, and where, is the loss that Royal Mail has suffered? Ernestmarples would never have bought a licence. The services that they scraped for it (which RM’s lawyers have demanded a list of) are free to the public.

Tom Watson is ever so slightly incandescent about this; if he were a light bulb you probably wouldn’t be able to buy him in the shops.

This is stupid: as Tom Watson points out, closing this service will also affect other services that are being offered. Not clever.