Feb 14, 2013

‘Banality of evil’: new documents lift the veil on data retention

New documents shed light on the enthusiasm of the Attorney-General's Department to move forward with (and think large on) data retention, and the resistance it encountered from industry.

Bernard Keane — Politics editor

Bernard Keane

Politics editor

Documents obtained under freedom of information reveal the Attorney-General's Department was further advanced in its preparations for a data retention régime than previously disclosed, and had a wider concept of the data to be retained than eventually accepted by the government. Data retention is one of 44 national security reform proposals currently under consideration by the Joint Committee on Intelligence and Security. The definition of the data to be retained under the proposal has been a vexed one for both the department, which came under fire from the committee for the poor quality of the paper outlining the reform proposals, and former attorney-general Nicola Roxon. She was forced to clarify that the proposal related to limited "traffic data" rather than any data that would reveal sites visited by individuals. Brendan Molloy of the Pirate Party this week obtained several documents relating to AGD's secret consultations with the telecommunications and internet industries in 2009 to 2012. While heavily redacted in parts, the documents afford a glimpse of the department's proposal and the mindset of the bureaucrats who gathered to discuss it. The documents cover three separate processes: the industry consultation process that commenced in mid-2009 and which was still going as late as May 2012 when Roxon referred the issue to JCIS; the ongoing interdepartmental discussion of data retention conducted by the Telecommunications Experts Group, an industry consultation co-chaired by AGD and the Department of Broadband and including the AFP, ACMA and (despite a redaction) ASIO; and a joint departmental-industry "privacy forum" in 2011 that dealt with aspects of data retention. The documents make clear that AGD originally had a much wider conception of data to be retained. In the initial consultation document circulated for the first meeting in November 2009 by Catherine Smith, the SES Band 1 officer who led the consultations, AGD specifies that telecommunications data includes "the identity of the sending and receiving parties", which necessarily includes destination addresses for each IP address. The purpose of the data is to enable agencies to "trace all communications from end to end". Over the subsequent two years, with industry consultations increasingly focusing on costs and industry consistently saying cost estimates would depend heavily on both the retention model and what data had to be retained, AGD appeared to shift position to focus retention on whatever data carriage service providers were already retaining for business purposes, in order to minimise costs. However, the department was careful to retain the big stick of compulsion. After one unidentified industry figure complained that data retention for business purposes changed regularly, reflecting the changing products that customers might switch to, and that industry should be compensated for having to retain data unnecessarily, Smith told them bluntly "the reality is that government puts forward proposals which industry may or may not agree with ... the reality is they have been reminded by law enforcement of the benefits to society of these particular policies (child safety, protection etc)." At least we know it's not only politicians who play the child p-rn card. Hitherto, we knew that AGD had drawn up draft legislation for data retention. The documents reveal that AGD was well advanced in preparing a Regulatory Impact Statement for the proposals, and in fact had begun preparing one as far back as 2009. The preparation of a RIS, which must be signed off by what is now called the Office of Best Practice Regulation and which includes an estimate of the likely financial impact on business, is a key step before a proposal goes to cabinet, and a RIS must accompany all legislation. Indeed, the notes from 2009 suggest AGD wanted to move quickly on the proposal, with industry only given a short time to comment on the discussion paper, and bureaucrats discussing options if industry didn't have time to respond comprehensively; for reasons unclear but perhaps electoral, the whole process then virtually ground to a halt in 2010 before resuming again the following year. The department was still trying to put together a RIS as late as February 2012; at another industry forum in February last year, an AGD director explained how different variables would affect the RIS's costing of the impact on business of the proposal. The documents also indicate the government knew all along that AGD was engaged in developing a data retention proposal and workshopping it with industry; Smith told a meeting of the Telecommunications Experts Group in June 2009 that she'd already written to the Attorney-General and (presumably via the AG) to the Prime Minister seeking approval to consult with industry on data retention. There was considerable industry pushback on the logistics of data retention. After two years of discussion, in 2011, the parties were still at loggerheads over costings, with industry saying they couldn't be clear about costs until AGD specified exactly what they wanted retained. One industry figure explained that IP addresses were increasingly useless as an identifier given the mobile nature of internet usage and suggested MAC addresses (misunderstood as MAP addresses by the department) as preferable. Another pointed out that privacy concerns about a single centralised data storage were "insurmountable" (AGD had initially proposed both centralised and decentralised data repositories). But the notes from the discussions between bureaucrats almost have a "banality of evil" tone to them, as public servants discuss a substantial government assault on privacy. Only one, ACMA member Chris Cheah (previously a long-time senior official from the Department of Broadband and a veteran of the telecoms policy), raised the obvious question: when was AGD planning to reveal its proposal and consult with the public, he asked at the start of the process in September 2009. AGD SES Band 2 officer Geoff McDonald replied that there'd be no public consultation until they'd finished developing a technical model and data sets. Cheah's question was right on the money: AGD never consulted with the public about data retention, not for another three years, until Roxon handballed the issue to JCIS for a public inquiry even as the department was still trying to get industry to commit to cost impacts so it could move the process forward within government. And even after all that time, AGD still hadn't finalised its "technical model and data sets". Or so it has insisted to both the public and the committee.

Free Trial

You've hit members-only content.

Sign up for a FREE 21-day trial to keep reading and get the best of Crikey straight to your inbox

By starting a free trial, you agree to accept Crikey’s terms and conditions


Leave a comment

9 thoughts on “‘Banality of evil’: new documents lift the veil on data retention

  1. robinw

    Bit of a worry isn’t it when the relevant bureaucrats don’t do the necessary homework to find out what a MAC address is. And then they compound the problem by misnaming it to MAP. And have these illuminati worked out the problems that users with a VPN would present to their scheme? I would expect not with the evidence to date. And what about those with a non Australian ISP? It boggles the imagination at the conniptions these could cause with the secret squirrel brigade.

  2. Mike Smith

    MAC addresses are hardly set in stone for a given NIC these days. It’s trivial to alter them.


  3. Mike Flanagan

    Thanks Bernard for another aricle to store in the memory bank.
    Be Warned, Be Aware and Be Bold, I suggest.
    With modern communication the bum shiners in the backrooms will soon become bored.
    Give ’em heaps Bernard.

  4. AR

    As Hannah Arendt pointed out, the people who are the most dangerous – by virtue (sic!) of their very inconsequentiality in the real world – are the inafequates who infest all structures, private or state, once the mass goes beyond a certain size.
    The Evil of Banality, people without navels, or vision, or EQ.

  5. Harry Rogers

    Once again Bernard you have given me confidence in journalism.

    I recall John Faulkner being dumb founded at the paucity of information given to his commitee by the AGD. Surely there must be some evil intent in what the AGD is trying to do and for what purpose? More to the point how the hell do these processes get initiated?

  6. Person Ordinary

    Bernard – Can you broaden the context for us? What forces are behind this push for “44 national security reform proposals?” Presumably its American, but which institutions are they using for leverage – IMF, secret services, something to do with the Free Trade Agreement? Surely it is not all home grown paranoia?

    Another interesting question might be how the activists go underground as the internet becomes just another weapon of the propagandists … is this related to the stop and search powers?

  7. Bernard Keane

    Person ordinary I think I covered this in a podcast at some point. The pressure comes from 3 areas: from agencies like AFP and ASIO that always want to expand their powers; from the Anglophone security establishment – intelligence and security officials in DC, Canberra, London, Ottawa and Wellington who work closely together and inevitably share ideas, and from the industry itself, which at a time when defence budgets are being cut sees lucrative opportunities in encouraging govts to spend more money on cybersecurity.

  8. Person Ordinary

    Bernard – Thanks, good stuff.

    I was hoping to see something broader like “counter-revolutionary zeal” but that may not impress the moderator, and it may be too early for many to see what is happening in those terms. We shall see …

  9. Dogs breakfast

    This looks like one of those classic examples of policy that comes back to bite one in the b__!

    So poorly thought out. Of course the AFP/ASIO set are going to want access to everything at all times in all circumstances.

    There are two ways around this, 1 is to monitor everything until there is so much damned data that asking for it will be impossible, and if theya re given all they want this will occur naturally.

    Or two, this will create a market in software to help hide your IP address, which will become more sophisticated as the spy set come closer to getting whatever they want.

    I’ve got nothing to hide, but I don’t believe government has the right to look at everything I do, watch, see, post etc.

    This has ‘perverse outcomes’ written all over it.

Share this article with a friend

Just fill out the fields below and we'll send your friend a link to this article along with a message from you.

Your details

Your friend's details