Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens

Archive for June, 2008

A “fetishistic” attitude to privacy is holding back crime mapping, says Heather Brooke

Saturday, June 28th, 2008

You may have heard of Heather Brooke: she’s one of the campaigners who got MPs’ expenses and spending put into the public eye through the Freedom of Information Act and a great deal of determination.

Now she’s written a terrific article for The Times about crime mapping, and why it’s needed:

The police in Britain, however, feel they “own” crime data and the public have no right to know what is happening. Yet access to criminal incident data is vital, as it allows the public to judge the effectiveness of the police and crime policies. In a void of ignorance, a politician or police chief can claim anything he likes about crime: that binge drinking is endemic or under control, that muggings are increasing or falling, that policing is working or failing.

The police can also hide their failings. Northumbria police claimed that only three crimes of note had occurred one weekend in May, yet a freedom of information request revealed that, in fact, there were more than 1,000 incidents, 161 of them violent.

As she points out…

Shamefully, the Information Commissioner has objected to the plan of Boris Johnson, the Mayor of London, to allow people to know what crime happens in their street, arguing that it would breach the privacy of the victims of crime. But the Data Protection Act does not prohibit personal information being disclosed. Its purpose is to ensure that such disclosure is for a legitimate purpose.

Yet again a policy that would be of great public benefit is being blocked by an unthinking, fetishistic attitude towards privacy. A balance can easily be struck between the privacy of those reporting crimes and the overall safety of citizens. The only people made safer by the current policy of wilfully enforced ignorance are poorly performing police chiefs.

That “fetishistic” sums it up perfectly. Just as “health and safety” has become the refrain for anyone trying to stop someone else doing something in the physical world, so “privacy” has become something that is used to block anything that might disrupt the status quo.

But disrupting the status quo is what Brooke, and indeed the Free Our Data campaign, are all about. It’s difficult, changing peoples’ mindsets: but if you keep pointing out that things are wrong about how it’s being done now, eventually you set up enough cognitive dissonance that something does happen. Brooke has done a huge amount to that end with MPs’ expenses (even though some are now trying to squeeze around it). We find common cause on data like this. The struggle continues.

Trading Funds review: terms of reference

Wednesday, June 25th, 2008

Things are starting to happen, through the work of the Power of Information taskforce.

Here’s the latest: the terms of reference for the study demanded by the POI report reviewing the trading fund model have just been published.

The study will be jointly led by Treasury and DBERR; Yvette Cooper and Baroness (Shriti) Vadera are the lead ministers, though the Cabinet Office POI taskforce secretariat will work closely with them. (We’re still waiting on the timescale.)

From the POI Taskforce blog:

the Taskforce will be assisting with this review, particularly looking at the value of the data held by the funds and whether the current business models and licencing arrangements are sustainable.

If you had to set our priorities what areas would you have us look at? Is there any research we should be looking at? Please kick off the discussion in the comments.

From the TORs themselves, part of the detailed work will include

With regards to a priority group of TFs (Ordnance Survey, Met Office, UK Hydrographic Office, Companies House, DVLA, and Land Registry) the exercise will aim to produce a detailed and definitive pricing and access policy for information held/created by TFs and the optimal constitutional structure for the several TFs to maximise benefit to the UK economy whilst maintaining public policy objectives.


changes in the pricing policy and licensing regime around information held by TFs may impact on their trading performance and value. Therefore for those TFs affected, the assessment will
  • provide an assessment of the business model or constitutional reforms needed to meet the Government’s commitment on access to information collected for public purposes by downstream markets; and
  • provide an assessment of the options for pricing the release of this information

We think this is going to be fun.

Cabinet Office Taskforce blogs… and you should read it

Wednesday, June 25th, 2008

Richard Allan and Tom Watson – the former a former Lib Dem MP who is now chair of the Power of Information Taskforce, the latter a Cabinet Office minister who set up the taskforce – have set up a blog for news of the progress that the taskforce is making.

It’s fascinating stuff, and we’ve been remiss in not noting more of it here.

For instance, there’s a proof-of-concept for crime mapping, including a presentation where the slides say things like “Is it safe to park my car here?” “Has crime in my area gone up or down?” and “How can I do something about it” – for as it points out, “Call to action [is] almost completely missing in existing mapping propositions”.

(On crime mapping, The Register is echoing the Times reporting today that the Information Commissioner thinks there are “privacy issues”. We feel sure we’ve heard this before – oh, because we actually looked into it, and found there isn’t a real objection. The comments on the Times piece are interesting, because they indicate that people *want* crime mapping, and think the objections of “the property industry” and the ICO are trivial. However one of the comments on the Register piece is interesting:

I’ve seen this … the Met Version. Working. With real data.

It had nothing to do with BoJo the clown. It was already being worked on for ages before, BoJo just made it a plegde and got lucky in the sense that it was already working before the election


The only thing that concerns me is that a visual representation will encourage house prices in certain areas to go into freefall as the red areas (high crime) are highlighted. Funnily enough anywhere near a train station (Underground or National) seems to have higher ratings than other parts of the boroughs but overall the map has average crime levels.

I suppose we will have to wait and see if this makes the police take less action in the red areas to reduce reported crime or tackle stupid things in the whole of London to raise the baseline average……


Which is a long diversion from the POI blog. Where you’ll find plenty of fascinating information, such as the old model for Parliamentary data:

and the new model such as can be used by theyworkforyou:

Get on and add it to your feeds. This is important stuff. And, even more importantly, it’s government being done transparently: when you can see what people are thinking, it helps you to influence it.

(Update: the comments feed isn’t obvious, but you can get it here er, here (thanks, Simon in the comments.)

Crime mapping coming more widely as government gets on board

Wednesday, June 18th, 2008

New guidance will mean that there will be more crime mapping: a paper published by the Cabinet Office and written by Louise Casey, the government’s crime adviser (the one, you’ll recall, who said that some anti-binge drinking schemes were nonsense) notes, inter (very many) alia, that

Police forces are due to provide standardised local information on crime, starting from Summer 2008, as part of the Government’s new crime strategy3. Some are already providing local information but what will be available from the Summer of 2008 is likely to remain highly variable. We hope police forces will draw on the evidence in this review to develop the information they provide over the next year.

In particular, there is strong public demand for consistency in the content and presentation of information about crime across the country and a strong focus on action. In a survey of members of the public for the review we found that:

• 72% of the public said the format of police websites should be the same across all police forces; and

• 87% wanted to see the same format used by all forces for the information they provide.

Beyond this, we believe there is scope for better presentation of comparative information on crime and the performance of the police and other criminal justice agencies which would be of interest to the public. With advances in mapping technology, there are several examples of crime information available on websites that allow the public to bring up crime information mapped onto a neighbourhood. [Emphasis added – CA]

Mapping and interactive reporting tools are useful and careful consideration should be given to their development and presentation. [Emphasis added – CA] We believe some consideration should also be given to standardising the information they provide on crime, based on best practice, so that consistent types of information are presented to the public in a recognisable and user-friendly format. While the focus of existing sites is local, some consideration should also be given comparisons between areas. An end aim could be to ensure that information is available on a national basis, consistent between areas. This would raise the profile of such information with the public – and a consistent format would make sense to a more mobile population. [Emphasis added – CA]

Jacqui Smith, the secretary of state at the Home Office, responded:

“We plan to publish monthly local crime data and we will take forward the report’s recommendations on local crime mapping and making sure every household receives ‘Crime Watch’ style information about the local fight against crime.

Not sure where that leaves police officers like Brian Paddick who think it’s all too unbearable to countenance the public seeing crime data. But the idea has now been so thoroughly floated, it’ll be next to impossible to simply bury it.

There are of course wrinkles. Stuart Grimshaw, who tipped us off to this announcement, has written a letter to his MP asking what formats the data will be provided in. Which matters, rather a lot. Do we want to scrape PDFs or websites? No, we want a decent XML feed, please. Not hard. Perhaps the police could be issued with geotagging systems – take a photo at the crime, send it with geotags (which can be done with GPS-enabled phones). Aggregate data, remove precision as required by class of crime. Send to web server with RSS output. Job done.

BBC’s iPM looks at crime mapping in Chicago

Tuesday, June 17th, 2008

The BBC’s iPM – the radio programme whose topics are suggested (though “chosen” would be overstating it) by its listeners – has looked at the topic of crime mapping, investigating how it’s done in Chicago.

There are two pieces: talking to Everyblock, the followup to chicagocrime,.org (which did the original crime mapping idea, building on the release by the Chicago police of the crime location data); and then talking to Jonathan Lewin, information services division commander in the Chicago police’s official crime mapping effort, and its GIS manager Joseph Kezon. You’ll have to visit the page to get the audio.

Notable points from the blog post:

Not everyone, of course, is happy with crime mapping. On the programme this weekend we’ll hear from groups who worry that crime-mapping could be counter-productive, affecting house prices, increasing fear of crime, and leading to areas being stigmatized.

Ah yes, the fear of knowing too much. Why don’t we just buy houses without ever seeing them? Why do we bother getting them surveyed? If house prices are affected, might it not also push up prices in places that don’t have crime? I’d have thought that more places would show low crime than will show high crime (because crime tends to cluster, for reasons that in hindsight are generally obvious), so that’s a net benefit for house prices; not that those should be the start or end of any conversation. (Please.)

Also amused by one of the comments:

I live in Surrey and we only have 10 police officers and all the police stations have been closed. They appear once a year for a major event and then we don’t see them again for 12 months. We do have a helicopter, loads of speed cameras, town centre cameras and a new video speed check van.

I think the police had a choice of more police on the beat or the helicopter and they decided that the helicopter looked like more fun. It hasn’t caught any criminals yet but they will produce statistics at the drop of a hat to assure you it is essential to crime busting.

The helicopter is very annoying and it would be useful to know the areas where it flies so potential house purchasers could avoid these areas. It flies around and around in pointless circles until residents are forced to report crimes in the next district so that it will fly away.

Don’t get me wrong, I don’t want more police on the beat. I want them to sell the helicopter and give me my money back.

Yup, helicopters are expensive to run. Maybe if there was a crime map you could decide where it should go more quickly..?

In The Guardian: surgeons’ deathrates online (but not for reuse)

Thursday, June 12th, 2008

The more things seem to change with government-collected information, the more they really stay the same. The latest example: surgeons’ deathrates, which will be made public under a new NHS scheme.

Except that, as we point out in NHS plans to reveal surgeons deathrates online in today’s Guardian, the data won’t be in a very usable form (at a guess, it’ll be a stack of PDFs – not even Excel files). And you’ll be banned from reusing it in any meaningful way.

It’s all of a part with the NHS Choices website, which will have the data:

NHS Choices is one of the government’s most lavish web projects, designed with web 2.0 very much in mind. Among other services, it promises “a social network for health”. A strategy published earlier this year says: “When people want to comment directly on their experiences of particular services, whether positive or negative, NHS Choices will become their first port of call.” The two-way information flow “will empower people to make informed decisions about their health and social care”.

However, while the strategy enthuses about the power of information in the new web world, it makes no mention of allowing re-use in mashups and commercial ventures. The site’s terms and conditions themselves suggest such use is out of bounds: “For your own personal non-commercial use you may copy, download, adapt or print off copies of the materials, information, data and other content included on NHS Choices (‘NHS Choices content’). You will need to obtain permission in writing from us before you make any other use of NHS Choices content.”

Now, let’s be clear that surgeons’ deathrates are easily misinterpreted. Someone who only ever does grommets isn’t going to have the sort of patient deathrate that someone doing open-heart surgery or brain surgery might. (There’s even an argument that what matters isn’t the operating-table mortality, but the 30- or 60-day mortality, since this tells you how well the patient recovered from everything.)

Nevertheless deathrates have a basic utility: it could have helped, for instance, to more quickly identify the Bristol heart babies’ abnormally high deathrates.

Then again, the BBC article linked to there is from 1998 – that’s ten years ago, folks – and in it, we’re told that “ministers believe the system [to make patient death rates at hospitals public, slightly different from this latest scheme] will become a powerful tool to raise standards and share information on the NHS.”

Kind of hard if people can’t reuse it easily. And also: “it could also work as an early warning system to prevent cases similar to that of Bristol.”

The data will need to have extra information, clearly, about what sort of operations were being done; simply saying “Surgeon X: deathrate Y” is inimical and useless. We’ll have to see..

APPSI to examine free data model, it says

Wednesday, June 11th, 2008

The Advisory Panel on Public Sector Information has put out its latest annual report. APPSI, you’ll recall, is now headed by David Rhind, the previous head of the Ordnance Survey (who also testified to the Treasury on the next census). But this report was signed off by Richard Susskind.

Kable has a short story on it:

Outgoing chair of APPSI, Professor Richard Susskind, said: “2007 was a pivotal year for the UK in relation to the re-use of PSI. Above all, we saw a marked increase across central government in the level of debate over the re-use of PSI. In particular, APPSI warmly welcomed the growing interest amongst ministers.”

The annual report itself is more interesting: re the Cambridge report, it says

APPSI can already confirm, however, that we welcome the tone and rigour of the Cambridge study – it is the kind of detailed and systematic economic analysis of trading funds and PSI re-use that we have been recommending since 2003; and we hope this represents the beginning of a new era of open and sophisticated thinking about the economics of PSI.

That’s encouraging. More transparency helps. And as for pressure upwards:

We intend, more frequently than we have in the past, to provide practical briefings to our Minister at the Ministry of Justice. These will cover key issues such as evidence, statistics and data relating to the impact of PSI; the governance of PSI and principles underpinning its re-use; the enforcement of the PSI Regulations; models and case studies clarifying the economics of PSI; the findings of ongoing horizon scanning by APPSI; and the adequacy and scope of information management activities across the public sector.

Basically, much more focus on on the economics of PSI. It also says it will “follow up” on progress from the recommendations of the reports into PSI such as The Power Of Information and the Cambridge study. And to that end…

To stimulate and widen debate about the future exploitation of PSI, we will conduct an initial inquiry into the implications of introducing a regime under which public bodies would be subject to some kind of obligation to make their PSI available for re-use. There is no such obligation today under the PSI Regulations.

Which is telling, isn’t it?

In the Guardian: how rows over intellectual property are causing problems for the 2011 census

Friday, June 6th, 2008

2001 Census: Royal Mail worker sorts envelopes
A Royal Mail worker sorts envelopes in the 2001 census. Copyright: The Guardian.

In this week’s Guardian Technology we look at the 2011 census – and beyond: the Commons Treasury Select Committee produced a report earlier this year pointing out that the £500m 2011 census is being hampered by rows between the Ordnance Survey, Post Office and local authorities about who owns the intellectual property in addresses, as the Office for National Statistics needs a comprehensive, accurate national address register to carry out its work.

The article, Traditional census is ‘obsolete’ also looks at what could follow: a rolling census carried out by noting peoples’ movements through address registers and so on.

An extract:

Angela Eagle, exchequer secretary to the Treasury, was pressed by Mark Todd, the Labour MP for South Derbyshire who sits on the committee, over the failure to create the register – particularly because, as he repeatedly pointed out, all the intellectual property lies within the public sector. Eagle responded that: “I would not underestimate the difficulty of the issues surrounding [a single national address register].” Todd suggested that a Gordian knot approach – cutting through the complexity at a stroke of legislation – might work. “We can all hope,” Eagle responded. But in the meantime, the government’s Office for National Statistics (ONS) will prepare its own register – a needless duplication.

The Free Our Data campaign would certainly back a Gordian approach: slice through the 10 years of rows by making the data available for free, without copyright restrictions, where they are gathered by government-owned organisations.

While Todd is no fan of the free data model, we can all agree with his frustration at the way that rows about data owned by public-sector organisations are holding back the development of another public-sector resource, an accurate census. The report recommends that the government “remove any outstanding obstacles to the production of an address register”.

But what are those obstacles? Only they know. Todd said: “I don’t know whether it’s the trading fund status of Ordnance Survey, legalistic barriers or failure of the will by government.”

You can find the Treasury select committee report here.

Is there a Valuation Office portal? No. Can we have one? No. Why not?…

Thursday, June 5th, 2008

An interesting exchange in the Commons between MarK Todd, who has some interest in matters of data availability (though we must point out he disagrees with the free data theorem – though, then again, he hadn’t finished reading the Cambridge economics report when we spoke to him earlier this week).

So here’s his first question (a written answer – Alastair Darling didn’t answer in the chamber)

Mark Todd: To ask the Chancellor of the Exchequer what arrangements exist for the Valuation Office Agency to make the council tax list for England and Wales available to third parties for value-added use; and if he will make a statement.

The written answer, from the Financial Secretary, HM Treasury:

Jane Kennedy: Council tax valuation lists for England and Wales are available to search on the Valuation Office Agency’s website at no cost. Full copies of lists are not otherwise available without a statutory gateway.

OK, then: on 16 May, another written PQ and answer:

Mark Todd: To ask the Chancellor of the Exchequer pursuant to the answer of 6 May 2008, Official Report, columns 817-8W, on council tax: valuation, whether a statutory gateway will be introduced.

Jane Kennedy: Statutory gateways exist in so far as they are provided by Commissioners for Revenue and Customs Act 2005 and other legislative provisions. This Department has no current plans to introduce further gateways.

I asked Mark Todd what the thrust of these questions was. He explained that he has constituents who are interested in finding out the value of houses in areas. They can get that data from local councils, in the form of the council tax valuations (which date back years). Except you can’t get it in bulk – unless you go in person to the council.

So there’s no obstacle to getting the information. The obstacle is to getting it in bulk. And the government isn’t going to make it easier to get. This is puzzling to Mr Todd. And to us, to be honest. There’s value there, but it’s being kept under wraps.

Why? The Post Office is in dire straits because too few companies are using its services to send letters. Companies that want to sell you stuff like addresses of people who fit their demographics. Yes, you can call it junk mail – but it’s easy to ignore. (I got a ton because I had a planning application on my house: lots of companies picked that up and sent me letters advertising their wares.) Even so, it’s all money for the Post Office.

Mr Todd will, we suspect, pursue this strange reluctance on the part of the government to find ways to make a bit of money from its information. He may think it should sell it. We think it would make better sense to make it free and let companies benefit from using it. But even if the ends are different, we have a common cause. We’ll watch this with interest.

Is the Information Commissioner against crime mapping? Actually, no

Monday, June 2nd, 2008

Mike Cross has got a statement from the Information Commissioner’s Office (ICO) on the proposed crime mapping by the new London mayor. Here it is in full:

“The ICO recognises the benefits of crime mapping to help police forces identify crime patterns, for instance to help plan the deployment of officers. However, when considering making such information public, it makes sense to think through what the intended benefits are and to weigh these against the potential privacy intrusion involved. We believe that this will involve carefully considering what level of detail, both of the offences themselves and where they took place, should be placed in the public domain. There is a risk that revealing exactly where a particular crime took place could lead to the identification of victims who may wish to remain anonymous. Even in circumstances where an individual cannot be confidently identified, there may be a risk that releasing information about a crime may lead some to infer the identity of the victim. It may also cause concern to those living in the locality, especially those with particular concerns about their security, for instance those living in some form of refuge. There is a need to balance the intended benefits of releasing such information against the potential intrusion or concern this might cause individuals.”

Points to note: the ICO does not come out against it, explicitly. It says that there is a need to balance benefits and risks. To be honest, it is vague to the point of pointlessness.

As to “revealing exactly where a particular crime took place”, that is done all the time by police reports (and then by media reports). And the proposals put forward by Boris Johnson (at least in his manifesto) are specific about *not* giving precise locations of crimes – it was clearly recognised. To reiterate, quoting the manifesto,

We believe there should be three kinds of map for each area for three categories of offences:

(a) where privacy is less of an issue: at their exact location with a pinpoint showing exactly where the offence occurred;
(b) where privacy might be an issue: identifying a 300 metre long street section within which an offence occurred; and
(c) where privacy is an issue: identifying the nearest whole street within which an offence occurred.

(Assaults would be in category (a); sexual attacks and domestic violence in (c), for example.)

It seems likely the ICO hasn’t looked at this, even though it’s been in the Tory manifesto, and of course here on our blog (aren’t we required reading for the ICO?).

So now it’s up to the police and, of course, to the mayor’s office. The clock is still ticking…

The police and crime mapping: and another perspective from the Observer

Sunday, June 1st, 2008

The Observer newspaper has an article today by Nick Cohen – a truly criminal approach to policing – which looks at the broader question of policing, and how setting targets can have a negative effect on other aims. It seemed worth quoting in the context of the earlier discussion about why the Met police appear to be against crime mapping (even if other regions have dipped their toes in).

Cohen writes:

Last week, Harriet Sergeant of Civitas described a police service which was close to incapable of doing its job. In a think-tank pamphlet, she delivered a devastating condemnation of an enclosed and self-referential bureaucracy which operated without regard to the wishes of the people who paid for it.

We now spend proportionately more than any other developed country on policing, she pointed out. The Home Office used targets to run it and delivered funding and bonuses to chief constables who filled its ‘sanction detention’ arrest quotas.

The first perverse consequence was that although the public expected the police to keep the peace, an officer who successfully stopped trouble was not rewarded because no trouble meant no arrests. More seriously, the police played the Home Office game by going for trivial offenders rather than serious criminals. Solving the case of a child who steals a Mars bar earned as many points as solving a murder. It made more sense to arrest rowdy children for ‘harassing a tree’ than to begin the hard work of tackling a potentially homicidal teenage gang.

Chris Dillow, author of New Labour and the Folly of Managerialism, describes Brown’s Mullettry as a marriage between Old Labour’s Fabian belief in the centralised state and Thatcherites’ worship of management consultants. Between them, they have spawned a bureaucracy which despises democratic accountability and, worse, does not and cannot work.

(“Mullettry” isn’t a reference to hairstyles, but to the senior officer to the fictional Inspector Frost.)

The whole article’s worth reading and considering while asking the question: would this police force want to release the data on crime mapping?