Ahead of the publication of the latest incarnation of the Communications Data Bill, due out tomorrow, I thought I’d take another look at how the UK compares with other countries when it comes to the number of requests for user data. There are many articles that quote the raw numbers and the US invariably comes out top in those terms – with a population of well over three hundred million people, that’s hardly surprising.

Instead, I look at the number of requests per million population – i.e, as a citizen how likely is it that someone such as the courts or police will be looking at my data. The UK has an unenviable record in this regard, topping the charts as the most-spied on population back in 2010. The UK remained top in Europe until the latter half of 2012, when France just managed to get ahead – a trend that was repeated in 2013.

Since then, more data has been made available. Originally, only Google produced data but many other companies have followed suit. The big three who publish useful data, in terms of requests received, are Google, (Including Google Plus) Microsoft (Including Skype) and Facebook. Below is the chart for the latter half of 2014, which is the latest period for which full data is available. France and the UK are still neck-and-neck on user data requests per million population, beaten only by Singapore’s recent obsession with collecting data from Google.

The European “right to be forgotten” has been in the news recently as it has been a little over 6 months since Google launched their formal process, allowing individuals to request the removal of search results for their name. But which Europeans are most and least likely to request removal from Google search results?

Germany, the UK, Italy and France all feature highly based on raw numbers, but that is to be expected given those are also the most populous countries. Looking at the numbers based on population the answer is, surprisingly Estonia. Unless some Estonians are submitting more than one request, which would be odd given one request can list multiple URLs to be removed, nearly one in every thousand Estonians have contacted Google requesting removal. At the other end of the scale, Bulgarians are the least likely to want to be forgotten with less than one request in ten thousand people with Greeks being close behind.

The UK comes right in middle of the pack. Despite the national stereotype, it appears we are no more shy than any other Europeans when it comes to having our details online.

Technical notes: Analysis is based on Google data as at 8th December, using World Bank population estimates from 2010. Countries with a population of less than 500,000 (Liechtenstein, Iceland) have been removed.

In the last few days, Reuters have produced a graph titled “Top 10 countries requesting for user data from tech companies“. It’s been circulated widely, having been picked up by Techspot, ISPreview, Gizmodo, io9 and others.

It is also very misleading because it reports the absolute number of requests, ignoring the relative size of the countries concerned. I have covered this before with Google’s transparency report, analyzing the data based on the population of the countries concerned. The Reuters graph goes beyond the Google data and includes the more recent Twitter and Microsoft/Skype transparency reports. So, I have repeated the same exercise and duplicated their graph with more representative numbers:

For comparison, here is the Reuters graph. (Click for a larger version)

There’s quite a difference – the US drops to number 10. Arguably, you could say that Luxembourg and Malta are unreliable data points due to their relatively small size, but that still puts the US at number eight. The raw data is reproduced below.

It’s that time again, when Google release their Transparency Report and I take a look at the per-capita rather than raw data.

The Google Transparency Report lists the number of requests via the courts or law enforcement, by country, for user data. These are often from governments but might also be in response to court orders – the only contry where this is broken down is the United States where The latest figures this time are for the second half of 2012 (2012 H2) so Google are now rather more up to date than they used to be, when they released data 6 months behind. There is remarkably little to report this time, with most countries maintaining their positions in the chart. Australia and France have swapped places, but remain remarkably close to each other, whilst the UK remains in fourth place.

What we do have now that we’ve not had previously is enough data to make it worthwhile charting the progress of Google user data requests over time. The upward trend in the number of requests per country is obvious, as is the fact that despite dropping to fourth place, there is still a steady rise in the number of requests in the UK – we have only dropped down due to other countries increasing their requests at an even more rapid rate.

Some welcome (On the surface) news today, that the UK is no longer the most spied upon population online, going by Google’s data. The most recent half-yearly Transparency Report shows that, when analysed per-capita, we’ve dropped down into 4th place.

(Requests refers to the number of user data requests submitted to Google in the half year period, per million people)

Sadly, the real story is a little more depressing. The number of requests in the first half of 2012 has remained relatively static at 23 requests per million people. The UK has lost it’s top spot purely because other countries – the US, France and Australia – have all increased their activity.

China, traditionally regarded as more authoritarian online, remains relatively low down the list coming in at 31st with just 0.1 requests per million. This is probably due to the lower internet usage in that country, coupled with greater state snooping allowing them to figure out users identity without involving Google.

Google released the latest update to their half-yearly Transparency Report today, something I’ve reported on previously. The Google data lists the number of “user data requests” per country, but what it doesn’t do is break it down per head of population.

Doing that breakdown gives depressing results. The UK had slipped into second place behind Singapore in terms of the number of requests per citizen, but as of the latest data – covering the second half of 2011 – we’re back on top. The table below shows the top ten countries, with the number of user data requests per million population. For comparison, I’ve also included the rankings of each country appearing in the top ten for the last couple of years since Google’s records began.

You can download the raw data (CSV) used to calculate this from the google figures too.

Remember, this is before the latest government plans come before parliament. We would probably drop off the charts completely if this legislation were to go through as they would no longer have to ask google for the data: The ISPs would be forced to do the snooping instead.

A while ago, I produced a per-capita analysis of the Google privacy data showing that the UK, per citizen, is the most snooped on country – there were more requests per person in the UK to Google than any other country. More up to date data is now available, for July to December 2010, showing that the UK is no longer the most snooped on – we’ve dropped to second place. Singapore does not appear in the January to June data but no reason is indicated for this.

(Note: Previous positions in the table below are based on latest data from Google. Some countries did not have data available at the time the previous blog post was compiled, so those numbers do not match)

And no real surprises in the removal requests category either as Lybia is still an order of magnitude ahead of every other country. The United States, home of free speech, drops out of the top 10 completely down to number 13 while the UK stays up at number six.

Google released their “transparency report” a couple of days ago. There’s been a little coverage of how high the UK is in the chart, but what I haven’t seen is an analysis of the data per head of population. It’s pretty logical that the US would have more requests than the UK, after all they have five times the population. So, here we have it, the top ten countries for government snooping per head of population. Data is for the 12 months from July 2009 to June 2010.

Removals are harder to count as the numbers are lower and start disappearing into noise, but for completeness, where data is available, here it is: