Yesterday, the first set of evidence into the Home Office’s controversial interception plans was heard in front of the special committee established to look at the draft bill and you can watch the Video on Parliament’s web site. (More is scheduled for this afternoon).

We learnt a few things about what’s being planned as a result of the evidence given, which was predominantly given by Charles Farr, ex-MI6 man and Director of the Office for Security & Counter-Terrorism.

Firstly, the existing Regulation of Investigatory Powers Act and Data Retention Directive are allowing police and security services to get access to around 75% of the data they are after. It’s envisaged that the wide-scale interception of communications data would increase that to 85% – so by only 10%, which seems a huge cost in both monetary and civil liberties terms for a relatively small increase. The existing shortfall was attributed in part to “ambiguities” in the EU Data Retention Directive as it’s implemented in the UK.

Secondly, when asked about their ability to break cryptography they Home Office mandarins ducked the question, instead saying that their preferred method was to “co-operate” with (I.e. coerce) service providers. This would be the likes of Google, Facebook and Twitter, both UK-based and foreign, so that they stored the communications data themselves.

They were quite clear on this point when asked about “black boxes” too and not just crypto – even though interception is the very first clause in the draft bill, they claim the main thrust is retaining data at the service provider.

A big hole in their argument as a result is that they have not made clear why altering the existing Data Retention Directive to allow this isn’t enough. There is a big difference in liberal terms between being asked to retain data you already have and actually listening in to obtain data.

The issue that remains is foreign non-cooperative service providers who cannot be coerced and the Home Office seems to imagine only intercepting communications as it enters and leaves the UK, and not widespread interception within the UK. This approach will cut the number of boxes they need. They may not even need to talk to big household-name service providers to do this, instead targeting the lesser-known (To the public) fibre providers who offer the bits of glass that go under the oceans, seas and English Channel.

This has the side effect of also intercepting private (Non-internet) traffic and communications transiting the UK from, say, the US to Germany. I’m sure this point hasn’t been lost on those pushing for it.

In terms of capability, the spooks believe it will be nearly impossible to remain anonymous with the volume of data they are able to collect, something that has sinister overtones for anyone with a genuine need to speak out against the establishment or against the police. You don’t even need to look as far as China to see this in action, as it would be the police justifying the use of interception and there is far from universal trust of the police to regulate themselves in this country.

On the topic of the police self-justifying their use of powers, requiring warrants to obtain data for lesser needs (e.g. Harassment and Non-payment of fines) was discussed and the Home Office did not seem to have a good reason why this shouldn’t be the case. Their argument in favour of allowing minor offences to be included is that they might escalate into more serious offences, and that’s OK because they don’t (ab)use these powers much. (Yet…)

Finally, they were asked by one MP if they could rule out “fishing” expeditions where they would obtain the data from hundreds of users but they were not able to do this. The example given was if they know a suspect was at a certain place, they might pull the communications data for everyone in that area at that time.

For those interested in this, there is also an ongoing consultation where you can submit evidence direct to the committee.

The draft Communications Data Bill has, at last, been published. We can finally debate what has been written down, rather than what the Home Office have been telling people in off-the-record briefings. Julian Huppert MP has an excellent post on safeguards which might be worth a look first, as those are the principles I would like to see in the Bill. Sadly, the draft bill falls down on several counts.

Firstly, we did point out quite forcefully in early debate that the police and security services were asking for powers that they did not have over the postal service. They’ve fixed that in the draft… by granting themselves powers over post too. Under the draft bill, the Royal Mail would need to scan and store the outside of every envelope that goes through the postal system if the Secretary of State asked them to.

Secondly, the vast majority of requests would still not require any form of judicial warrant. Instead, the police would still retain the ability to authorised themselves to go after communications data.

Finally, (for the major concerns), clause 1 which places the obligations on ISPs to collect data is still far too broad. “Interception” is not allowed, but that would seem to only rule out real-time monitoring as it uses the previous RIPA definition. ISPs could still be mandated to look at the content of all traffic to try to drag out “communications data”.

Internet traffic is not like the post, with the addressee neatly written on the outside. Instead, the outer envelope (IP) contains another envelope (TCP). You need to collect together all the IP envelopes in order to make sense of the TCP conversation. Once you have that, you need to open the TCP envelopes to see if they contain little Instant Message, Club Penguin, World of Warcraft or Facebook envelopes. Then, we need to read the data off that envelope, no mean feat given that World of Warcraft envelopes will be written in whatever language makes sense to them, not to us as service providers.

By the time you’ve built this system, even assuming you figure out how, you have something that is required to read the entire content of everyone’s communication to figure out where the envelopes stop and the letters start.

All this is before anyone puts wax seals on their envelopes encrypts their data, which I suspect will start happening quite widely should this bill pass.

How are the Home Office going to do this anyway? Black Boxes. Clause 1(2) allows the Home Office to impose “requirements for telecommunications operators… to acquire, use or maintain specified equipment or systems“. The Home Office might not operate the black boxes, but by mandating the supplier they’re not far off having complete control. I rather suspect ISPs will have very limited information on or access to any mandated systems, which will limit technical oversight.

Even then, Labour’s original “central database” idea isn’t far off, courtesy of clauses 14 to 16 which talk about “filtering” systems operated not by service providers but by the Home Office. There are no safeguards proposed to stop the Home Office from simply demanding all data held by an ISP as part of a trawl for interesting information.

There are a few other holes that need addressing too, but I would expect them to be tightened up in the usual course of events. For example, if you’re given a notice saying your data might be needed for a court case, you have to keep it until you are told it is no longer needed. However, there is no provision, requirement or obligation for the scope of the retention to be limited. Given how long court cases can take, this could mean that an ISP ends up storing all of it’s communications data for years.

It still needs a little more scrutiny. I notice they’ve slipped in powers to allow snooping to collect unpaid fines and taxes, but I forget if that’s still in RIPA. No doubt as people pour over this more, we’ll get better and better breakdowns of what it all means.