Free Our Data: the blog

A Guardian Technology campaign for free public access to data about the UK and its citizens

Archive for 2008

A “fetishistic” attitude to privacy is holding back crime mapping, says Heather Brooke

Saturday, June 28th, 2008

You may have heard of Heather Brooke: she’s one of the campaigners who got MPs’ expenses and spending put into the public eye through the Freedom of Information Act and a great deal of determination.

Now she’s written a terrific article for The Times about crime mapping, and why it’s needed:

The police in Britain, however, feel they “own” crime data and the public have no right to know what is happening. Yet access to criminal incident data is vital, as it allows the public to judge the effectiveness of the police and crime policies. In a void of ignorance, a politician or police chief can claim anything he likes about crime: that binge drinking is endemic or under control, that muggings are increasing or falling, that policing is working or failing.

The police can also hide their failings. Northumbria police claimed that only three crimes of note had occurred one weekend in May, yet a freedom of information request revealed that, in fact, there were more than 1,000 incidents, 161 of them violent.

As she points out…

Shamefully, the Information Commissioner has objected to the plan of Boris Johnson, the Mayor of London, to allow people to know what crime happens in their street, arguing that it would breach the privacy of the victims of crime. But the Data Protection Act does not prohibit personal information being disclosed. Its purpose is to ensure that such disclosure is for a legitimate purpose.

Yet again a policy that would be of great public benefit is being blocked by an unthinking, fetishistic attitude towards privacy. A balance can easily be struck between the privacy of those reporting crimes and the overall safety of citizens. The only people made safer by the current policy of wilfully enforced ignorance are poorly performing police chiefs.

That “fetishistic” sums it up perfectly. Just as “health and safety” has become the refrain for anyone trying to stop someone else doing something in the physical world, so “privacy” has become something that is used to block anything that might disrupt the status quo.

But disrupting the status quo is what Brooke, and indeed the Free Our Data campaign, are all about. It’s difficult, changing peoples’ mindsets: but if you keep pointing out that things are wrong about how it’s being done now, eventually you set up enough cognitive dissonance that something does happen. Brooke has done a huge amount to that end with MPs’ expenses (even though some are now trying to squeeze around it). We find common cause on data like this. The struggle continues.

Trading Funds review: terms of reference

Wednesday, June 25th, 2008

Things are starting to happen, through the work of the Power of Information taskforce.

Here’s the latest: the terms of reference for the study demanded by the POI report reviewing the trading fund model have just been published.

The study will be jointly led by Treasury and DBERR; Yvette Cooper and Baroness (Shriti) Vadera are the lead ministers, though the Cabinet Office POI taskforce secretariat will work closely with them. (We’re still waiting on the timescale.)

From the POI Taskforce blog:

the Taskforce will be assisting with this review, particularly looking at the value of the data held by the funds and whether the current business models and licencing arrangements are sustainable.

If you had to set our priorities what areas would you have us look at? Is there any research we should be looking at? Please kick off the discussion in the comments.

From the TORs themselves, part of the detailed work will include

With regards to a priority group of TFs (Ordnance Survey, Met Office, UK Hydrographic Office, Companies House, DVLA, and Land Registry) the exercise will aim to produce a detailed and definitive pricing and access policy for information held/created by TFs and the optimal constitutional structure for the several TFs to maximise benefit to the UK economy whilst maintaining public policy objectives.


changes in the pricing policy and licensing regime around information held by TFs may impact on their trading performance and value. Therefore for those TFs affected, the assessment will
  • provide an assessment of the business model or constitutional reforms needed to meet the Government’s commitment on access to information collected for public purposes by downstream markets; and
  • provide an assessment of the options for pricing the release of this information

We think this is going to be fun.

Cabinet Office Taskforce blogs… and you should read it

Wednesday, June 25th, 2008

Richard Allan and Tom Watson – the former a former Lib Dem MP who is now chair of the Power of Information Taskforce, the latter a Cabinet Office minister who set up the taskforce – have set up a blog for news of the progress that the taskforce is making.

It’s fascinating stuff, and we’ve been remiss in not noting more of it here.

For instance, there’s a proof-of-concept for crime mapping, including a presentation where the slides say things like “Is it safe to park my car here?” “Has crime in my area gone up or down?” and “How can I do something about it” – for as it points out, “Call to action [is] almost completely missing in existing mapping propositions”.

(On crime mapping, The Register is echoing the Times reporting today that the Information Commissioner thinks there are “privacy issues”. We feel sure we’ve heard this before – oh, because we actually looked into it, and found there isn’t a real objection. The comments on the Times piece are interesting, because they indicate that people *want* crime mapping, and think the objections of “the property industry” and the ICO are trivial. However one of the comments on the Register piece is interesting:

I’ve seen this … the Met Version. Working. With real data.

It had nothing to do with BoJo the clown. It was already being worked on for ages before, BoJo just made it a plegde and got lucky in the sense that it was already working before the election


The only thing that concerns me is that a visual representation will encourage house prices in certain areas to go into freefall as the red areas (high crime) are highlighted. Funnily enough anywhere near a train station (Underground or National) seems to have higher ratings than other parts of the boroughs but overall the map has average crime levels.

I suppose we will have to wait and see if this makes the police take less action in the red areas to reduce reported crime or tackle stupid things in the whole of London to raise the baseline average……


Which is a long diversion from the POI blog. Where you’ll find plenty of fascinating information, such as the old model for Parliamentary data:

and the new model such as can be used by theyworkforyou:

Get on and add it to your feeds. This is important stuff. And, even more importantly, it’s government being done transparently: when you can see what people are thinking, it helps you to influence it.

(Update: the comments feed isn’t obvious, but you can get it here er, here (thanks, Simon in the comments.)

Crime mapping coming more widely as government gets on board

Wednesday, June 18th, 2008

New guidance will mean that there will be more crime mapping: a paper published by the Cabinet Office and written by Louise Casey, the government’s crime adviser (the one, you’ll recall, who said that some anti-binge drinking schemes were nonsense) notes, inter (very many) alia, that

Police forces are due to provide standardised local information on crime, starting from Summer 2008, as part of the Government’s new crime strategy3. Some are already providing local information but what will be available from the Summer of 2008 is likely to remain highly variable. We hope police forces will draw on the evidence in this review to develop the information they provide over the next year.

In particular, there is strong public demand for consistency in the content and presentation of information about crime across the country and a strong focus on action. In a survey of members of the public for the review we found that:

• 72% of the public said the format of police websites should be the same across all police forces; and

• 87% wanted to see the same format used by all forces for the information they provide.

Beyond this, we believe there is scope for better presentation of comparative information on crime and the performance of the police and other criminal justice agencies which would be of interest to the public. With advances in mapping technology, there are several examples of crime information available on websites that allow the public to bring up crime information mapped onto a neighbourhood. [Emphasis added – CA]

Mapping and interactive reporting tools are useful and careful consideration should be given to their development and presentation. [Emphasis added – CA] We believe some consideration should also be given to standardising the information they provide on crime, based on best practice, so that consistent types of information are presented to the public in a recognisable and user-friendly format. While the focus of existing sites is local, some consideration should also be given comparisons between areas. An end aim could be to ensure that information is available on a national basis, consistent between areas. This would raise the profile of such information with the public – and a consistent format would make sense to a more mobile population. [Emphasis added – CA]

Jacqui Smith, the secretary of state at the Home Office, responded:

“We plan to publish monthly local crime data and we will take forward the report’s recommendations on local crime mapping and making sure every household receives ‘Crime Watch’ style information about the local fight against crime.

Not sure where that leaves police officers like Brian Paddick who think it’s all too unbearable to countenance the public seeing crime data. But the idea has now been so thoroughly floated, it’ll be next to impossible to simply bury it.

There are of course wrinkles. Stuart Grimshaw, who tipped us off to this announcement, has written a letter to his MP asking what formats the data will be provided in. Which matters, rather a lot. Do we want to scrape PDFs or websites? No, we want a decent XML feed, please. Not hard. Perhaps the police could be issued with geotagging systems – take a photo at the crime, send it with geotags (which can be done with GPS-enabled phones). Aggregate data, remove precision as required by class of crime. Send to web server with RSS output. Job done.

BBC’s iPM looks at crime mapping in Chicago

Tuesday, June 17th, 2008

The BBC’s iPM – the radio programme whose topics are suggested (though “chosen” would be overstating it) by its listeners – has looked at the topic of crime mapping, investigating how it’s done in Chicago.

There are two pieces: talking to Everyblock, the followup to chicagocrime,.org (which did the original crime mapping idea, building on the release by the Chicago police of the crime location data); and then talking to Jonathan Lewin, information services division commander in the Chicago police’s official crime mapping effort, and its GIS manager Joseph Kezon. You’ll have to visit the page to get the audio.

Notable points from the blog post:

Not everyone, of course, is happy with crime mapping. On the programme this weekend we’ll hear from groups who worry that crime-mapping could be counter-productive, affecting house prices, increasing fear of crime, and leading to areas being stigmatized.

Ah yes, the fear of knowing too much. Why don’t we just buy houses without ever seeing them? Why do we bother getting them surveyed? If house prices are affected, might it not also push up prices in places that don’t have crime? I’d have thought that more places would show low crime than will show high crime (because crime tends to cluster, for reasons that in hindsight are generally obvious), so that’s a net benefit for house prices; not that those should be the start or end of any conversation. (Please.)

Also amused by one of the comments:

I live in Surrey and we only have 10 police officers and all the police stations have been closed. They appear once a year for a major event and then we don’t see them again for 12 months. We do have a helicopter, loads of speed cameras, town centre cameras and a new video speed check van.

I think the police had a choice of more police on the beat or the helicopter and they decided that the helicopter looked like more fun. It hasn’t caught any criminals yet but they will produce statistics at the drop of a hat to assure you it is essential to crime busting.

The helicopter is very annoying and it would be useful to know the areas where it flies so potential house purchasers could avoid these areas. It flies around and around in pointless circles until residents are forced to report crimes in the next district so that it will fly away.

Don’t get me wrong, I don’t want more police on the beat. I want them to sell the helicopter and give me my money back.

Yup, helicopters are expensive to run. Maybe if there was a crime map you could decide where it should go more quickly..?

In The Guardian: surgeons’ deathrates online (but not for reuse)

Thursday, June 12th, 2008

The more things seem to change with government-collected information, the more they really stay the same. The latest example: surgeons’ deathrates, which will be made public under a new NHS scheme.

Except that, as we point out in NHS plans to reveal surgeons deathrates online in today’s Guardian, the data won’t be in a very usable form (at a guess, it’ll be a stack of PDFs – not even Excel files). And you’ll be banned from reusing it in any meaningful way.

It’s all of a part with the NHS Choices website, which will have the data:

NHS Choices is one of the government’s most lavish web projects, designed with web 2.0 very much in mind. Among other services, it promises “a social network for health”. A strategy published earlier this year says: “When people want to comment directly on their experiences of particular services, whether positive or negative, NHS Choices will become their first port of call.” The two-way information flow “will empower people to make informed decisions about their health and social care”.

However, while the strategy enthuses about the power of information in the new web world, it makes no mention of allowing re-use in mashups and commercial ventures. The site’s terms and conditions themselves suggest such use is out of bounds: “For your own personal non-commercial use you may copy, download, adapt or print off copies of the materials, information, data and other content included on NHS Choices (‘NHS Choices content’). You will need to obtain permission in writing from us before you make any other use of NHS Choices content.”

Now, let’s be clear that surgeons’ deathrates are easily misinterpreted. Someone who only ever does grommets isn’t going to have the sort of patient deathrate that someone doing open-heart surgery or brain surgery might. (There’s even an argument that what matters isn’t the operating-table mortality, but the 30- or 60-day mortality, since this tells you how well the patient recovered from everything.)

Nevertheless deathrates have a basic utility: it could have helped, for instance, to more quickly identify the Bristol heart babies’ abnormally high deathrates.

Then again, the BBC article linked to there is from 1998 – that’s ten years ago, folks – and in it, we’re told that “ministers believe the system [to make patient death rates at hospitals public, slightly different from this latest scheme] will become a powerful tool to raise standards and share information on the NHS.”

Kind of hard if people can’t reuse it easily. And also: “it could also work as an early warning system to prevent cases similar to that of Bristol.”

The data will need to have extra information, clearly, about what sort of operations were being done; simply saying “Surgeon X: deathrate Y” is inimical and useless. We’ll have to see..

APPSI to examine free data model, it says

Wednesday, June 11th, 2008

The Advisory Panel on Public Sector Information has put out its latest annual report. APPSI, you’ll recall, is now headed by David Rhind, the previous head of the Ordnance Survey (who also testified to the Treasury on the next census). But this report was signed off by Richard Susskind.

Kable has a short story on it:

Outgoing chair of APPSI, Professor Richard Susskind, said: “2007 was a pivotal year for the UK in relation to the re-use of PSI. Above all, we saw a marked increase across central government in the level of debate over the re-use of PSI. In particular, APPSI warmly welcomed the growing interest amongst ministers.”

The annual report itself is more interesting: re the Cambridge report, it says

APPSI can already confirm, however, that we welcome the tone and rigour of the Cambridge study – it is the kind of detailed and systematic economic analysis of trading funds and PSI re-use that we have been recommending since 2003; and we hope this represents the beginning of a new era of open and sophisticated thinking about the economics of PSI.

That’s encouraging. More transparency helps. And as for pressure upwards:

We intend, more frequently than we have in the past, to provide practical briefings to our Minister at the Ministry of Justice. These will cover key issues such as evidence, statistics and data relating to the impact of PSI; the governance of PSI and principles underpinning its re-use; the enforcement of the PSI Regulations; models and case studies clarifying the economics of PSI; the findings of ongoing horizon scanning by APPSI; and the adequacy and scope of information management activities across the public sector.

Basically, much more focus on on the economics of PSI. It also says it will “follow up” on progress from the recommendations of the reports into PSI such as The Power Of Information and the Cambridge study. And to that end…

To stimulate and widen debate about the future exploitation of PSI, we will conduct an initial inquiry into the implications of introducing a regime under which public bodies would be subject to some kind of obligation to make their PSI available for re-use. There is no such obligation today under the PSI Regulations.

Which is telling, isn’t it?

In the Guardian: how rows over intellectual property are causing problems for the 2011 census

Friday, June 6th, 2008

2001 Census: Royal Mail worker sorts envelopes
A Royal Mail worker sorts envelopes in the 2001 census. Copyright: The Guardian.

In this week’s Guardian Technology we look at the 2011 census – and beyond: the Commons Treasury Select Committee produced a report earlier this year pointing out that the £500m 2011 census is being hampered by rows between the Ordnance Survey, Post Office and local authorities about who owns the intellectual property in addresses, as the Office for National Statistics needs a comprehensive, accurate national address register to carry out its work.

The article, Traditional census is ‘obsolete’ also looks at what could follow: a rolling census carried out by noting peoples’ movements through address registers and so on.

An extract:

Angela Eagle, exchequer secretary to the Treasury, was pressed by Mark Todd, the Labour MP for South Derbyshire who sits on the committee, over the failure to create the register – particularly because, as he repeatedly pointed out, all the intellectual property lies within the public sector. Eagle responded that: “I would not underestimate the difficulty of the issues surrounding [a single national address register].” Todd suggested that a Gordian knot approach – cutting through the complexity at a stroke of legislation – might work. “We can all hope,” Eagle responded. But in the meantime, the government’s Office for National Statistics (ONS) will prepare its own register – a needless duplication.

The Free Our Data campaign would certainly back a Gordian approach: slice through the 10 years of rows by making the data available for free, without copyright restrictions, where they are gathered by government-owned organisations.

While Todd is no fan of the free data model, we can all agree with his frustration at the way that rows about data owned by public-sector organisations are holding back the development of another public-sector resource, an accurate census. The report recommends that the government “remove any outstanding obstacles to the production of an address register”.

But what are those obstacles? Only they know. Todd said: “I don’t know whether it’s the trading fund status of Ordnance Survey, legalistic barriers or failure of the will by government.”

You can find the Treasury select committee report here.

Is there a Valuation Office portal? No. Can we have one? No. Why not?…

Thursday, June 5th, 2008

An interesting exchange in the Commons between MarK Todd, who has some interest in matters of data availability (though we must point out he disagrees with the free data theorem – though, then again, he hadn’t finished reading the Cambridge economics report when we spoke to him earlier this week).

So here’s his first question (a written answer – Alastair Darling didn’t answer in the chamber)

Mark Todd: To ask the Chancellor of the Exchequer what arrangements exist for the Valuation Office Agency to make the council tax list for England and Wales available to third parties for value-added use; and if he will make a statement.

The written answer, from the Financial Secretary, HM Treasury:

Jane Kennedy: Council tax valuation lists for England and Wales are available to search on the Valuation Office Agency’s website at no cost. Full copies of lists are not otherwise available without a statutory gateway.

OK, then: on 16 May, another written PQ and answer:

Mark Todd: To ask the Chancellor of the Exchequer pursuant to the answer of 6 May 2008, Official Report, columns 817-8W, on council tax: valuation, whether a statutory gateway will be introduced.

Jane Kennedy: Statutory gateways exist in so far as they are provided by Commissioners for Revenue and Customs Act 2005 and other legislative provisions. This Department has no current plans to introduce further gateways.

I asked Mark Todd what the thrust of these questions was. He explained that he has constituents who are interested in finding out the value of houses in areas. They can get that data from local councils, in the form of the council tax valuations (which date back years). Except you can’t get it in bulk – unless you go in person to the council.

So there’s no obstacle to getting the information. The obstacle is to getting it in bulk. And the government isn’t going to make it easier to get. This is puzzling to Mr Todd. And to us, to be honest. There’s value there, but it’s being kept under wraps.

Why? The Post Office is in dire straits because too few companies are using its services to send letters. Companies that want to sell you stuff like addresses of people who fit their demographics. Yes, you can call it junk mail – but it’s easy to ignore. (I got a ton because I had a planning application on my house: lots of companies picked that up and sent me letters advertising their wares.) Even so, it’s all money for the Post Office.

Mr Todd will, we suspect, pursue this strange reluctance on the part of the government to find ways to make a bit of money from its information. He may think it should sell it. We think it would make better sense to make it free and let companies benefit from using it. But even if the ends are different, we have a common cause. We’ll watch this with interest.

Is the Information Commissioner against crime mapping? Actually, no

Monday, June 2nd, 2008

Mike Cross has got a statement from the Information Commissioner’s Office (ICO) on the proposed crime mapping by the new London mayor. Here it is in full:

“The ICO recognises the benefits of crime mapping to help police forces identify crime patterns, for instance to help plan the deployment of officers. However, when considering making such information public, it makes sense to think through what the intended benefits are and to weigh these against the potential privacy intrusion involved. We believe that this will involve carefully considering what level of detail, both of the offences themselves and where they took place, should be placed in the public domain. There is a risk that revealing exactly where a particular crime took place could lead to the identification of victims who may wish to remain anonymous. Even in circumstances where an individual cannot be confidently identified, there may be a risk that releasing information about a crime may lead some to infer the identity of the victim. It may also cause concern to those living in the locality, especially those with particular concerns about their security, for instance those living in some form of refuge. There is a need to balance the intended benefits of releasing such information against the potential intrusion or concern this might cause individuals.”

Points to note: the ICO does not come out against it, explicitly. It says that there is a need to balance benefits and risks. To be honest, it is vague to the point of pointlessness.

As to “revealing exactly where a particular crime took place”, that is done all the time by police reports (and then by media reports). And the proposals put forward by Boris Johnson (at least in his manifesto) are specific about *not* giving precise locations of crimes – it was clearly recognised. To reiterate, quoting the manifesto,

We believe there should be three kinds of map for each area for three categories of offences:

(a) where privacy is less of an issue: at their exact location with a pinpoint showing exactly where the offence occurred;
(b) where privacy might be an issue: identifying a 300 metre long street section within which an offence occurred; and
(c) where privacy is an issue: identifying the nearest whole street within which an offence occurred.

(Assaults would be in category (a); sexual attacks and domestic violence in (c), for example.)

It seems likely the ICO hasn’t looked at this, even though it’s been in the Tory manifesto, and of course here on our blog (aren’t we required reading for the ICO?).

So now it’s up to the police and, of course, to the mayor’s office. The clock is still ticking…

The police and crime mapping: and another perspective from the Observer

Sunday, June 1st, 2008

The Observer newspaper has an article today by Nick Cohen – a truly criminal approach to policing – which looks at the broader question of policing, and how setting targets can have a negative effect on other aims. It seemed worth quoting in the context of the earlier discussion about why the Met police appear to be against crime mapping (even if other regions have dipped their toes in).

Cohen writes:

Last week, Harriet Sergeant of Civitas described a police service which was close to incapable of doing its job. In a think-tank pamphlet, she delivered a devastating condemnation of an enclosed and self-referential bureaucracy which operated without regard to the wishes of the people who paid for it.

We now spend proportionately more than any other developed country on policing, she pointed out. The Home Office used targets to run it and delivered funding and bonuses to chief constables who filled its ‘sanction detention’ arrest quotas.

The first perverse consequence was that although the public expected the police to keep the peace, an officer who successfully stopped trouble was not rewarded because no trouble meant no arrests. More seriously, the police played the Home Office game by going for trivial offenders rather than serious criminals. Solving the case of a child who steals a Mars bar earned as many points as solving a murder. It made more sense to arrest rowdy children for ‘harassing a tree’ than to begin the hard work of tackling a potentially homicidal teenage gang.

Chris Dillow, author of New Labour and the Folly of Managerialism, describes Brown’s Mullettry as a marriage between Old Labour’s Fabian belief in the centralised state and Thatcherites’ worship of management consultants. Between them, they have spawned a bureaucracy which despises democratic accountability and, worse, does not and cannot work.

(“Mullettry” isn’t a reference to hairstyles, but to the senior officer to the fictional Inspector Frost.)

The whole article’s worth reading and considering while asking the question: would this police force want to release the data on crime mapping?

Why the police are against crime mapping – and what it tells us

Friday, May 30th, 2008

The BBC’s Today programme reported this morning that the Information Commissioner has, at least temporarily, held up the plan by the new London mayor Boris Johnson to introduce crime mapping to London. Crime maps, you’ll recall, were one of Johnson’s manifesto pledges. (Note: I can’t find anything to this effect on the ICO news page – anyone got a link?)

Apart from wondering why the manifesto pledge wasn’t taken for a quick spin by Johnson and the Tories past those who might have an interest (such as the Information Commissioner), the justification given afterwards by Brian Paddick, a former deputy assistant commissioner at the London Metropolitan Police (and the Liberal Democrat candidate for the mayor’s job), was illuminating. Why? Because it showed how deep the thinking that “we know better than you” runs inside British administration.

(You can hear the clip here; requires Real Player.)

Paddick said that the police already use crime mapping internally, and that from time to time the police chiefs for various areas would be called together to account for why they weren’t doing so well.

And how effective were those meetings in getting anything done? “It depended who was chairing them,” Paddick said. Good chair = police chief shamed, or driven, to do something. Bad chair = not much happens.

So why was he against it? “There are signif downsides to that [public crime mapping] process, potential to stigmatise areas, to create ghettos… and to make underreporting ofcrime even worse – people not reporting.”

The following is a rough, first-pass transcript.

Q: But it’s just telling people where crime happens.
Paddick: But we should be focusing on wht will improve efecti of police. The New York success of accountability didn’t come through making the crime data available to the public, but to Comstat, where police commanders were compared to their peers in open forum. We have tried that in London wher have half dozen commanders – asked those who do better what the secret is so they can tell the others…. The police already use crime mapping data themselves, using it in a sophisticated way. The only difference between what happens and what Boris Johnson is suggesting is that of making it public.

Q So the police already have mapping street by street to decide where to deploy resources – all Boris Johnson is suggesting is to make it public.
Paddick: yes… there are systems to hold police commanders accountable though meeetings. Making crime maps available down to street level is a lot of pain for very little gain.

Q But if we do what New York did, why might we not get better results?
Paddick: It has to be said that Comstat process – that is, holding local police commanders to account in one room and account for why crime had gone up or down – whether or not that worked depended who was chairing the meeting. It’s not a very British thing to hold people to account in front of their peers. It had mixed results depending on how the chair held them to account. (Emphasis added – CA.)

Q I thought it was about using it in clever ways, overlaying demographics, the location of porn pawn [thanks Stuart in comments] shops, to lead to more effective deployment of police?
Paddick: …. yes there is potential to use crime mapping more effectively to deploy resources, but the basic principle is available to the police and they are using it at present.

Well, wouldn’t the court of public opinion possibly be quite a good chair for the biggest possible meeting? Accountability is an uncomfortable process, yes. We see again and again that organisations dislike it. MPs don’t like having their expense claims made public – and when they are, they fight against them. In the same way, the police probably don’t like the idea that their full-time (one hopes) efforts to stop and solve crimes might be overlooked by the people to whom it’s actually happening.

But the fact is that this is a process that is happening in society, and any organisation that tries to ignore it does so at its peril because it loses the trust of the public. Journalism (which is a topic I know about) is going through this process: readers can cross-check what we write, point out where we make errors, bring new information we didn’t have. The principle that I try to work on with blog posts like this one is that when I write it, I know more than the average reader. But that implies that half the readers know more about the topic already – and if they can be encouraged to pitch in (as, happily, people do here) then we all benefit, because we have even more information.

The police may not like the idea that their work is made visible, and some people may feel uncomfortable with the idea of knowing about crimes being reported in their vicinity. (Then again, house prices are falling fast enough anyway. Crime maps won’t make a difference.) But the idea that the police are smarter about crime and criminal patterns than all of the population who might be looking at a crime map is, frankly, insulting. The average police officer is probably much smarter about crime than most of us. But it’s that group of the wider population that can find patterns that they can’t who they should be recruiting. If getting police commanders together in a room makes them o better, try making the room so big it covers the country. Then we’ll see some change.

Postscript: Wait, what’s this? In a story from earlier this month, with Gordon Brown pushing all sorts of policy initiatives, we find this:

Mr Brown will say: “My aim is to ensure we utilise all the innovation at our disposal to improve public services in this country and to give more power to those who use them.”

He is to praise government “successes” such as introducing broadband into every school, electronic border controls and electronic data records in hospitals.

There will also be pledges to push ahead with neighbourhood “crime mapping”, video identification of suspects, electronic school report cards, and online GP appointment booking.

(Emphasis added – CA.)

Oh, and a bonus link from Heather Brooke’s website (for she it is who has fought so hard to get MPs’ expenses made public): Police PR spending:

We found that police forces across the UK are spending £39m each year on press and PR – enough to fund an extra 1,400 full time officers and more than enough to cover the annual police pay rise withheld by the Government. The force at the top of the league (Police Service Northern Ireland) spends eight times more per person on PR than the lowest (Derbyshire). Meanwhile, forces spend nearly ten times more on PR (what police want us to know) than on FOI (what we want to know).

In today’s Guardian: how Inspire was watered down

Thursday, May 22nd, 2008

German floodInspire sounded like it would be such a great idea: data would be swapped for free between governments, everyone would benefit.

But practice and theory tend not to be next-door neighbours. And so it has proved with Inspire, which as this week’s Guardian story – An Inspired debate about access – points out, hasn’t lived up to early hopes.

First, some very good news. Civil servants revealed last week that the British government has begun work on a system to make all the geospatial data it holds on the natural environment available for free inspection and re-use. Now the bad news. In this context, “free” means we will still have to pay to download much key data, especially if it is to be published or otherwise used commercially.

The British government has until May 2009 to “transpose” Inspire’s measures to UK legislation: the Department for Environment, Food and Rural Affairs (Defra) is leading the work. In a culture where public agencies jealously guard the intellectual property in their databases, and often have to recover the costs of running them, the new regime may face some opposition.

“It is clear that Inspire will require organisations to collaborate better,” David Lee, one of the Inspire project team at Defra, said at the Public Sector conference in London last week. He revealed that the department is considering creating a geoportal, which would allow anyone to track down any geospatial data set, regardless of who owns or maintains it. “The vast majority of transactions to members of the public will be free,” Lee said.

However it is not quite as simple as that. While the Inspire directive requires “metadata” (machine-readable data about data) to be provided free of charge, access to the databases themselves may be charged, especially if it is for commercial use.

And so that dichotomy continues: commercial use versus “private” use. Everyone, it seems, should have Chinese walls in their minds – how else can you be sure that knowledge gained in one’s personal life won’t be exploited in their commercial existence?

Free Our Data: now on Twitter

Thursday, May 15th, 2008

You may or may not have heard of Twitter, which creates a social network (sorry) around sending SMS-length messages via the web or mobile. The idea is that you can send messages to the world in general, or communicate publicly with friends, or privately with friends. You can think of it as real-time micro-blogging (ie, very short but to the minute).

If you want to understand more about the service (which is free), you can read Making the most of Twitter, at the Guardian.

But the point here is that there is a Free Our Data twitter account, which will principally be used for notifying when there are new blog posts here. (We tend to note any stories on the Guardian too, though I may add that to the feed.) It shouldn’t be overwhelming. And we’ll aim to follow those who follow us, so that we know what’s going on.

You’ll find the page at (Congratulations Ian Sealy – our first follower!)

In today’s Guardian: the problem with route data for glider pilots

Thursday, May 15th, 2008

Today in the Guardian, with Access to data needs to take off, we look at the slightly unusual (in the context of this campaign) case of glider pilots, in particular, and the hassles that they have in trying to load flight data, in particular about where they may and may not fly.

The wrinkle being that the data is available – but not in a convenient machine-readable format that they could load into their GPS-based systems. And responsibility for deciding whether to do that is split between the Civil Aviation Authority – which is a government-appointed regulator – and National Air Traffic Services, a privatised organisation in which the government is the largest minority shareholder, along with commercial airlines and staff.

The point being, if you want someone to do something that would streamline things, where do you press?

Britain must love flying. How else to explain the fact that the space over the north of London is, as Andrew Watson puts it, “one of the most complicated bits of three-dimensional airspace on the planet”? And as the spokesman for National Air Traffic Services (Nats), which looks after air traffic control for commercial services, notes, there’s also the south of England, home to commercial airports such as Heathrow, “executive” airports such as Biggin Hill and Farnborough, and 60-odd other airports used by pilots of light aircraft, parachutists, hang-gliders and gliders. It’s very, very busy.

But as Watson, a glider pilot, points out, that busy-ness leads to a peculiar conflict over data. Specifically, useful access to data about where you may and may not fly in the UK. And with about 10,000 glider pilots and 3,000 gliders in the UK making some 500,000 flights a year and able to range across the country during a single flight – that access to data matters.

The Civil Aviation Authority (CAA) sets the rules about who can fly where. As Watson explains: “The UK is covered by a three-dimensional pattern of airspace into which various kinds of aircraft may or may not be allowed to enter.” All pilots must carry a chart showing that 3D space if they fly more than five miles from their airfield. (In practice, they all carry the authoritative CAA chart, published annually.)

That might seem straightforward – except that the boundaries can change from week to week or even day to day. That means pilots must check the CAA and Nats websites for the “Notices to Airmen” (or Notams, which presumably apply to female pilots too) about temporary airspace restrictions.

What Watson finds perplexing is that though you need to know those airspace “boundaries” – which are unrelated to commercial mapping data – the CAA does not publish machine-readable versions.

It’s a fiendishly complex situation, though what’s really puzzling is why NATS and the CAA, which would benefit from glider (and other private) pilots being able to get this data easily into their machines (without the possibility of transcription errors), preventing dozens of intrusions into controlled airspace every year, don’t do it.

Then again, it’s a funny world, flying. As I learnt while researching it, Nats and the CAA don’t accept GPS as a navigation device. Why? “Because it’s intentionally degraded,” I was told (by Nats). That is, your position isn’t perfectly accurate. But only to 10m, surely? Or even 100m? Is that really going to make all the difference when you’re thousands of feet (or metres) up?