Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens


Archive for 2006

Interview re FOD campaign on ‘Meme Therapy’

Wednesday, July 26th, 2006

I was interviewed by the Meme Therapy blog (site? news organisation? whatever). Mostly it’s a recap of the thinking behind the campaign; read it here.

Comments are welcome, here, there, or on your own blog.

Guardian writeup of RSA debate

Thursday, July 20th, 2006

Today’s Guardian Technology has a writeup of the RSA debate (which should be available in MP3 format here according to the Open Access blog – I haven’t checked yet, though).

If you couldn’t make it, well, we are hoping that we’ll have something along the same lines later this year, perhaps in the autumn. Assuming that we haven’t persuaded every minister in sight of the rightness of our cause by then :-)

Ed Parsons blogs his thoughts on the RSA debate

Wednesday, July 19th, 2006

There’s a very interesting post on the blog of Ed Parsons (he’s the chief technology officer at Ordnance Survey) about his view of the debate on Monday evening. Worth reading in its entirety; this part in particular stuck out for me.

  • RSA debate – my thoughts..
    in all countries, the funding of geographic information is never high on any politicians agenda, look at the chronic underfunding of the USGS for example. This is not the case interestingly I would suggest for statistical information, which perhaps more directly is seen to drive policy ?

My question would be, how do we get government to realise that geographical information drives policy in a million tiny but essential ways?

(Seen at edparsons.com)

OPSI finds against OS in row over AddressPoint licensing

Tuesday, July 18th, 2006

[Update: if you’ve come here via the Guardian article on spam, then first, apologies; secondly, the location you actually want to go to is http://noc.net.umd.edu/cgi-bin/netmgr/whoami – and that will tell you what IP address you’re on. Again, sorry for the error, which was introduced during editing.]

We’ve seen the OPSI report into Intelligent Addressing’s complaint (148K PDF) about the lack of transparency in the AddressPoint licensing terms.

Essentially, Intelligent Addressing (hereafter IA) says that OS seems to have a different licencing regime internally than externally: that a company outside OS can’t get access to AddressPoint on anything more than one-year contracts, whereas OS can use AddressPoint indefinitely. That puts external organisations like IA, which wants to license AddressPoint in order to compile the National Land and Property Gazeteer (NLPG), at a disadvantage.

And OPSI agreed.

Para 35:

In addition, the Complainant alleges that the lack of transparency extends to the website in that it is not clear how the Framework
Agreement interacts with the specific use contracts. Furthermore, the
Complainant states that there are few published details about the
partner licensing options.

Para 37:

The PSIH states that the website contains significant openly accessible
information, but does not publish the licensing options as the majority of
partners have actively managed accounts, and the PSIH works closely
with them to discuss their tailored licensing solutions.

The PSI Regulations also state that “Where conditions are imposed they
shall not unnecessarily restrict (a) the way in which a document can be re
-used or (b) competition� (Regulation (12) (1)). It appears to OPSI that the
terms of the licence unnecessarily restrict the way in which AddressPoint
can be re-used and unnecessarily restrict competition since such terms
are unnecessarily prescriptive. Some conditions imposed by the PSIH
restrict the way in which AddressPoint can be used and this restricts
competition between the Complainant and the PSIH since it immediately
puts the Complainant’s products which use AddressPoint at a
disadvantage to those of the PSIH. One such condition relates to time, as
the PSIH can grant licences with a longer term to end-users than others
such as the Complainant, who is limited to twelve month end-user
licences as licensees want more security in the length of time they are
able to use the information. This means that the Complainant is at an unfair disadvantage. Another example is cost as the Complainant has
to pay a higher licence fee for the use of AddressPoint. OPSI does not
consider that such restrictions are necessary.

You can read the full complaint and response. But here’s OS’s response, hot off the press:

“On the OPSI report, as Carol Tullo [head of OPSI] was saying last night [at the RSA/Free Our Data debate], it was only published last Thursday and so we still need to discuss it with OPSI in the context of the points we are already exploring from the reverification process you reported on in March.

That said, we do have a number of concerns around matters of fact and law in the report and are considering the options open to us including our right of appeal to the Advisory Panel on Public Sector Information (APPSI). The report purports to make wide-ranging recommendations regarding our licensing policy on the basis of a narrow investigation of a single, and atypical, set of circumstances.

Please be aware also that although they have a right to make a complaint to OPSI, Intelligent Addressing are not a licensee of Ordnance Survey information; they are a contractor to Local Government Information House, who are the licensee. We believe that OPSI’s interpretation of some of the concepts, terminology and clauses within Public Sector Information regulations could make it difficult for us to properly differentiate between the terms and conditions used for products and services licensed directly to end users for their internal business operations, from those applying to partners who generate commercial revenue by exploiting the data in their own products and services (often by adding value to our data); for example, a direct utility customer using the data to manage multi-million pound assets of pipes and cables would have to be licensed in a similar way to a small publisher using the data to produce consumer products retailing at a few pounds.

OPSI also appear to ignore the fact that third party data plays an increasingly important role in the content of our products and services, bringing added value and benefit to users. Royal Mail has a significant IPR interest in the products involved in the complaint. OPSI’s reading of relevant PSI regulation clauses could result in us having to implement pricing and licensing terms and conditions that would undermine the commercial interests of these third parties. This could in turn discourage them from offering their data into collaborative products and inhibit our ability to produce marketable data or to generate adequate revenue for reinvestment.

Our adoption of specific use contracts followed wide consultation with licensed partners from whom it gained broadly-based support. It enables pricing of data to be varied according to end-user perceptions of value for clearly identifiable areas of use. With this mechanism in place it guarantees that competing partners licensing products into the same market have paid similar licensing costs to us. The model assists in creating competitive markets among partners providing similar products and services and also enables users with specific needs to acquire their information at appropriate prices. Introducing a “one size fits all� approach may also be anti-competitive where others have already entered high cost/high value markets, and it may unreasonably inhibit more from doing so.

We are committed to honouring our full obligations as a Trading Fund with delegated authority to protect and exploit Crown Copyright and in so doing we are required to operate on a commercial agenda. We are proud to be accredited under the Information Fair Trader Scheme and are working with OPSI to ensure our licence terms and conditions remain open, transparent and fair.”

RSA/Free Our Data debate draws big crowd

Tuesday, July 18th, 2006

Encouragingly, scores of people braved the summer heat – and the lack of air conditioning in the main hall – to come to the Royal Society for the Encouragement of Arts, Manufactures and Commerce (RSA) in London for the RSA/Free Our Data debate on public sector information and copyright.

There is a webcast (21MB MP3) and a transcript (220K PDF). In the meantime, if you have any comments lingering from the event, please add them here. You don’t have to register or give a full name to comment.

In today’s Guardian: who will address the postcode mess?

Thursday, July 13th, 2006

In today’s Technology Guardian, Michael Cross examines the Royal Mail’s postcode address file – such a valuable item, if it could only be used well.

Royal Mail

is coy about how much the Postcode Address File costs to maintain and how much it receives from licensing, or even whether it runs at a profit.

Licensing arrangements for the Postcode Address File are only one part of the addressing imbroglio. Even if the Royal Mail were to give the database away to all comers, addressing would still be messy.

For a start, the postcode file has big gaps. According to Barr, it holds only 60% of buildings in England – the Royal Mail is not interested in structures such as churches, which do not receive mail. Because of the purpose for which they were set up, postcodes may bear little relevance to reality – the initial component, the “post town”, relates to the nearest sorting office rather than the nearest town.

As Ed Parsons (chief technology officer of Ordnance Survey) noted on this blog, the flaws mean that there are three databases in operation – the RM PAF, the OS’s address layer, and the National Land and Property Gazetteer (I think that’s right on the latter).

And why?

This chaos arises directly from the UK government’s policy of encouraging state-owned bodies, which are usually monopolies in their fields, to treat information as an asset to be exploited commercially. Free Our Data argues that this resource should be funded by taxation and made available for free to all takers, to stimulate a vibrant knowledge economy.

Some heads really need to be knocked together, we think.

Will the Post Office and Ordnance Survey ever agree about house names?

Monday, July 10th, 2006

Part of the reason the previous post was late arriving was because I moved house last week. It went successfully, but here’s an annoyance. In the last house, we changed the name (we didn’t like the old one) – notifying the local council, which accepted the name and put it into its records. Clearly, the council notified the Post Office, which accepted the name… except that it changed the ending of the name, which was “House”, to “Cottage”.

Now, in no way was our house a cottage. It was a two-storey semi-detached early 1900s building. Clearly, the misnaming was one of those intentional errors that any database compiler puts in so that it can spot people stealing its data (for there’s no copyright in facts, but there is copyright in fiction, so if you add some false data to a database…)

And now our new home has a name, which differs from the Post Office’s name, which differs too from the OS’s name.

I can understand that they want to have proprietary ownership of their data, but really, this is ridiculous. The upshot is that when we call people to tell them our new address, if they’re an organisation using the Postcode Address File (PAF), they only want the first line of the address and the postcode. But when you tell them the property name and the postcode, there’s a pause while they try to figure out whether you’re right or they’re right.

It’s an annoyance, though it’s indicative too of the absurd lengths that the organisations have to go to in order to “protect” their precious data. At the cost of creating hassle for the rest of us..

Who owns patient records? The patient, the GP, the health service or its contractors?

Monday, July 10th, 2006

(Apologies for the delay in posting this; I’ve been moving house – on which more in another post.)

The Guardian looks at the vexed question of Who owns patient records? Vexed, because it’s not clear quite who owns the information (and hence access to the information) on those records. Rather as with banks, it looks as though while health organisations agree that you own the data about you, they’re going to charge you to access it, because it’s on their “property”.

In the days when records were on paper… a rule of thumb was that the secretary of state for health owns the paper; the GP the ink; and the patient the information. In the computer era, he says, the picture can be confused by ownership of equipment.

Dr Richard Fitton, a GP who routinely allows patients to see their electronic records, says the government should state outright that patients own their data. “If you pay me through taxes to produce a record I think that philosophically these records belong to you. If you want me to publish them on the web, or give you a complete copy, that’s fine.” Fitton dismisses as rubbish the idea that patients should be spared clinical details. “If you’re dying of cancer, you want to see everything.”

The piece has already drawn some letters, parts of which I’ll reproduce here:

The concept of who ‘owns’ electronic records isn’t a useful one. It’s more useful to consider who has a legitimate right to have access to them. Health data are created when the system providing care interacts with the recipient of that care – and both then have rights with respect to the data.

If I get my car serviced by a garage then both I and the garage need to have access to the record of the event. How would the garage owners be expected to react if I told them I wasn’t going to let them use the record on their computer?

If you are going to use the NHS you need to accept that it has to use your information to run the system (with appropriate systems to ensure the maximum privacy obtainable of course). To pretend otherwise is ingenuous.
Dr Rod Muir, Consultant in Public Health, Information Services Division (ISD), NHS National Services Scotland, Edinburgh

And another:

I think your article is wrong, at least semantically. Under the “access to medical records act”, GP’s can charge a maximum fee of £50 for a *copy* of a patient’s medical records. I don’t think GP’s can charge a fee a patient to read their records. They can charge a fee for interpretation of medical information.

What patients don’t realize is that when ticking a box for consent for access to their medical records when making an accident injuries claim for example, they are allowing their “agent” to have a copy of their *whole* medical records, not just the events around the accident. I have had to contact a number of patients to clarify this matter with them and most are quite shocked that *everything* is usually asked for. I have had to check with patients regarding child protection issues, sexually transmitted diseases, etc which are of course unrelated to road traffic accidents.
Dr David J Plews, GP Medical Adviser Rotherham PCT

Thanks to both for their input. There’s also been some blog action in response; see these Technorati links for blogs which mention the piece.

Why is the government trying to corner the market for travel-direction sites?

Thursday, June 29th, 2006

In today’s Guardian we ask why it is that Transport Direct is using lots of data from local and central government (paid for by local and central taxes) to provide a service that has bugs – and is entering the market after a number of private ones.

In Time to tell this travel site where to go, Michael Cross investigates Transport Direct, which has had three million users this calendar year and uniquely offers point-to-point directions.

In an age when it is not seen as appropriate for the public sector to run power stations or railways, why is it running nationalised industries in what should be the most dynamic sector of all, the web-based knowledge economy? The question lies at the heart of our campaign, which argues that government’s role should be to collect and administer high-quality raw data, but make it freely available to everyone to create innovative services.

Transport isn’t the first area where the government has come late into such offerings:

Since its conception nearly a decade ago, “e-government” has been exempt from conventional political wisdom about competition, monopoly and state aid. The consequences are not only theoretical. In 2000, at the height of the dotcom boom, a London startup company called iMPower had the idea of launching a service to sell fishing licences on the web. It was supposed to usher in a new age of “intermediaries” providing electronic routes to public services. In theory, this was supported by government policy – but another government policy required the Environment Agency to launch its own fishing licence service on the web. The private-sector offering was unable to compete.

[Transport Direct’s chief executive Nick] Ilsley says that research by the department before Transport Direct’s launch showed the private sector wasn’t interested in providing a one-stop all-purpose site. However the site was launched in a market already populated by the private sector, albeit with less sophisticated offerings. Their operators argue that these are more in tune with public needs.

Our question: why not just make the feeds available for anyone to make use of, and build sites which could compete with each other, which would benefit taxpayers by generating revenues, rather than costing them for a function that sits on top of essential government?

How much does it cost to display a map online? The correct(ed) answer

Thursday, June 29th, 2006

After last week’s blog post in which we suggested it would cost millions to put an OS map of the UK online if you had a successful site (with, say, 2,000 users per day), we’ve had lots of people saying that’s wrong. And it is.

So today’s Guardian carries the story How much does it cost to display an OS map on a website?:

Much less than we estimated last week. In “Time to account for travel maps’ costs” (June 22), we wrote: “For a charity to put [those maps] on a webserver that might be used by hundreds of people (a typical server can handle 2,000) would cost millions of pounds annually.”

In fact, Ordnance Survey points out, the cost would be more like £18,000 per year – a hundredfold less than we suggested.

Of course we should have checked and checked again. Yup. We were following mySociety’s expectation that if it cost £1,000 for a single user internally per year, that that would scale up scarily once you displayed the data online.

But the confusion is understandable. OS’s page on copyright licensing for internet use (at http://www.ordnancesurvey.co.uk/oswebsite/business/copyright/purpose/internet.html) does not specify that internet users en masse only count as one; hence mySociety’s expectation that internet use would lead to an explosion in costs.

Not so, said OS. For a single scale mapping of the country – say, at 1:50,000 scale – including Code Point for finding postcodes (so you can get a map of a postcode’s location), the annual licence for a website serving 20,000 map images per day, every day, would be £18,200 per year.

To provide the mapping service most web users are used to, one would have to license several scales: we have grown used to being able to zoom in on a point.

Streetmap offers seven mapping scales; Multimap offers 13, though not all appear to come from OS. Using multiple scales will, of course, ramp up costs very quickly – as will being popular. But even licensing seven scales will only take your annual costs to around £100,000 – not into the millions. Unless, of course, you are wildly successful.

So, that’s corrected, we hope. But even so:

Tom Steinberg, mySociety’s director, says: “The price for these maps, which cover only a small chunk of the country, is way above affordability by most small and medium-sized enterprises, a group that employs more than half the UK workforce. Furthermore, it excludes the entire caste of internet-based enthusiasts who’ve produced about 80% of all the innovative mapping work in the world over the past couple of years.”

Even at the new lower price, we probably won’t be putting those maps on soon..

Mark the date: July 17 at the RSA, London, for the Free Our Data debate

Thursday, June 22nd, 2006

There will be a public debate on July 17 at the Royal Society for the Encouragement of the Arts (RSA) in London on the issue of

  • Derek Wyatt MP, chairman of the all-party group on the internet, to chair the debate
  • Paul Crake, RSA programme director

  • Carol Tullo, director, Office of Public Sector Information, controller of HMSO, and Queen’s printer
  • Ed Parsons, chief technology officer, Ordnance Survey
  • David Vaver, director of the Oxford University Intellectual Property Research Centre
  • Charles Arthur, editor, Guardian Technology
  • Tickets are free. You can apply online at http://www.rsa.org.uk/events/detail.asp?eventID=1901 [link updated – thanks!] or phone the RSA lectures booking line on 020 7451 6868.

    See you there!

    How does Ordnance Survey justify its licensing costs when its accounts are disputed?

    Thursday, June 22nd, 2006

    In this week’s Guardian Technology supplement we have Time to account for travel maps’ costs, which looks at the costs of Ordnance Survey licences. If the organisation is right about the importance of its work (it claims to underpin £100 billion of economic activity), then – as Tom Steinberg of mySociety points out – even a slight error in overpricing the licence could mean a huge fall in tax revenues.

    That is illustrated by the cost of a licence for “time travel maps” that mySociety created. As the article points out,

    How much would it cost to put those maps on a webserver that anybody could access? MySociety asked Ordnance Survey’s licensing department. It calculated that displaying 16 “map tiles” with the relevant data would cost between £837.81 (for a 1:25,000 scale) or £1,032.71 (for a 1:250,000 scale).

    Does that sound good? Here’s the sting in the OS’s small print: “All prices for one user, one-year licence and exclude VAT. Terms of data use are internal business use, display and promotion as long as there is no financial gain.” So for a charity to put that on a webserver that might be used by hundreds of people (a typical server can handle 2,000) would cost millions of pounds annually.

    And here’s the other thing: the National Audit Office does not accept the OS’s accounts, and has not done since it became a trading fund in 1999, because OS treats the National Geographic Database (which originated with taxpayers’ money) as an intangible asset – but puts no value on it.

    That in turn means that the £9.2m surplus the OS shows was almost four times above its target return, of 5.5% on capital employed (tangible and intangible assets). Yet put in the NAO’s estimate of the NGD’s value, and you still get a rate of return that’s about double the 5.5%.

    What does that imply? To us, that OS licences are expensive – they’re generating too high a rate of return. Read and see what you think.

    OECD meeting on public sector information: now online

    Saturday, June 10th, 2006

    The OECD recently held a meeting in Paris on public sector information and charging models. We were represented (ably, by Mike Cross) but haven’t yet digested it in its fullness.

    However, you can find most of the presentations and topics at http://www.oecd.org/document/17/0,2340,en_2649_34223_36860241_1_1_1_1,00.html. If you notice any particular elements that you think deserve to be highlighted, add them in the comments..

    Carol Tullo on free data: ‘what’s the point?’

    Thursday, June 8th, 2006

    Today’s Guardian has a piece in which the writer managed to corner Carol Tullo of the Office of Public Sector Information, to put the question: why not make data free?

    “Why should we be gatekeepers? We have enough to do in our day jobs than to worry about what the local economy may find interesting.”

    The default position of government should be to trade in information, Tullo said, adding that transparency and openness benefits government in many ways. She cited the non-political website TheyWorkForYou.com – which repurposes data from Hansard online to let users find out about MPs’ voting records, attendance and even register of interests – as an example of how making government information available can benefit society.

    “The people at TheyWorkForYou.com have said to me, ‘we shouldn’t be providing this [site]. This is something government should have been providing.’ Actually, no. This is a perfect example of entrepreneurial private-sector activity,” Tullo said.

    Except that the people at theyworkforyou.com point out that they’re not private-sector; they’re activists, showing what ought to be done. And they had to break the law in order to set up the site.

    Read more at Make it work for us, Ms Tullo

    How the Met Office lost millions of pounds trying to compete with the private sector

    Thursday, June 1st, 2006

    One of the arguments of this campaign is that it’s a mistake for the public sector to get into fields where it is handicapped by the nature of its structure – that is, in selling data commercially. (The public sector is ideally placed to collect and collate data: it has laws and tax-raising powers, for example. And lots of time, often.)

    This week, an example of how the Met Office came unstuck in 2001 – and lost £4.5 million of our money doing it – in this week’s Technology Guardian, over the failure of WeatherXchange, a joint venture with private organisations.

    Extract:

    [The present Met Office chief executive Mark] Hutchinson told the Commons defence select committee on May 23 that his organisation was dominated by public-service civil servants, and will need to bring in relevant expertise in order to build its commercial activities. “We don’t have an awful lot of hard, private-sector commercial experience,” he said.

    The day before, Peter Ewins, the chief executive of the Met Office from 1997 to 2004, told the committee he did not believe other participants in WeatherXchange made initial investments but had instead provided credibility and expertise. “That sounds incredibly naive and amateurish,” said Kevan Jones, a Labour MP. “If you had done that in local government, you’d have been shot.”

    Ewins, who remained chairman of WeatherXchange after leaving the Met Office, says the joint-venture was hit by poor European growth in weather derivatives. But he added: “After I left the Met Office, there was in my view not the champion of that relationship that was so necessary to its success.”

    And here’s the kicker:

    According to [MP Kevan] Jones, “the reason why the relationship broke down was the fact that the Met Office realised how lucrative this venture was, and was selling information directly to the market, rather than going through WeatherXchange.”

    Responding, Ewins said: “You have pre-empted by about 10 seconds what I was going to go on to say.”

    Got that? The Met Office undermined the company it had invested in, because it could do better by selling data directly to the market. Sure, it shouldn’t have started the company. But the worrying thing is that if the company had been a success, then the Met Office would have preferred to supply that than other private-sector organisations – which would start to create a private monopoly where none existed. So in some ways, it’s a blessing that WeatherXchange failed. But really, it should never be public money involved in these ventures. Risking money in the private sector is the job of banks. Providing data is the job of the public sector. Perhaps some people will start to listen. Does anyone have Kevan Jones’s email?