Mar 18, 2015

Your guide to the data retention debate: what it is and why it’s bad

It's time to clarify what data retention is and the nature of the threat it poses to citizens.

Bernard Keane — Politics editor

Bernard Keane

Politics editor


Leave a comment

12 thoughts on “Your guide to the data retention debate: what it is and why it’s bad

  1. Dogs breakfast

    Great isn’t it.

    I wonder why Tony isn’t describing this as a great new government spying on every citizen tax!

  2. David Vaile

    Great stuff.

    As to what metadata is, not only is it still effectively undefined, and open to change without return to Parliament, the proponents have knowingly or incompetently mislead us about what the current plan needs to collect – see Geoff Huston (chief scientist APNIC, former chief scientist Telstra, former secretary, Internet Society), the posts of August 2014 and May 2013.

    He explains that in the modern world of “Carrier Grade NAT” — such as now necessary for mobiles, and soon to be needed for other access methods, after we exhausted the IPv4 address space in 2011 — the new combination of concurrent sharing of an IP address and dynamic fast re-allocation of IP addresses means the minimum necessary Internet connection metadata will be 33,000 samples a day down to millisecond accuracy: more than a thousand points an hour or 20 million per 2 year retention period.

    And even if you only want a user’s “IP” and identity, you will have to also collect every server IP address as well (the web domain can easily be looked up from its IP).

    A far cry from just “your IP address”.

    So, while the proponents may not “want” to collect your web browsing history, at the bare minimum it is increasingly technically necessary to collect the web server IP address for every element of everything you (and everyone) clicks or views, even if the aim is merely to be able to identify you from among the several other people who may be sharing your IP address at any instant.

    A IP is not yet a complete Web address (URL), but with the server address of the host page, and and a similar IP fingerprint of all the elements, images, cookies and ads on it, this gives the lie to apparent assurances that “your web browsing will not be tracked”.

    If Geoff is correct (and you’d want a globally authoritative technical source to refute him), even if the aim is just to identify you, on a CG-NAT system they cannot afford NOT to track every server you browse, every service and every element you use. Without this server-end IP logging, identification is typically not possible under CG-NAT.

    The Joint Committee showed little comprehension of this, and no interest to pursue the implications, or the explanation for why no proponent was willing to acknowledge it, or to explain what it means: even without “content”, and not “wanting” to track your browsing, the unintentional metadata by-catch often just happens to be pretty close to a browsing history, down to the millisecond level.

  3. Pamela Lyon

    Bravo, Crikey, for putting the issues involved in data retention in such clear perspective for all of us, not just the media.

    However, I would like to take issue with one statement in this piece: “Australia’s agencies generally have a better record on not abusing their powers than foreign agencies.”

    We simply don’t know that. Moreover, we have no reason to suppose it is true. In fact, we have every reason to believe that it is not, simply because Australians’ access to information about the workings of government–and in particular the intelligence services–is extremely circumscribed compared to, say, the United States.

    The only reason we know of contemporary abuses by the NSA, CIA and FBI in the USA–and in Australia, for that matter–is because of Edward Snowden, William Binney (also former NSA) and others who were willing to trash their lives so that we could have this debate at all.

    The Abbott Government, with the collusion of Labor, has done as much as humanly possible to ensure a Snowden- or Binney-like series of disclosures never happens here. Do you think that would be the case were it true that Australian governments, which are subject to much lower levels of public oversight (due in part to a complacent media), are somehow more immune to abuses of power than other countries? Really?

    The only reason Edward Snowden and other US intelligence service whistleblowers are willing to do what they do is because of the Bill of Rights to the US Constitution–rights they believe in –and the knowledge that abuses have happened before.

    They know abuses have happened before because of the ‘Church Committee’, headed by Senator Frank Church (D-Idaho), which investigated illegal actions by the US intelligence services in 1975. The safeguards passed into law following this investigation have been the subject of (sometimes incremental, sometimes wholesale) modification ever since, and particularly since 9/11, much as the legal restrictions placed on the US financial sector following the Great Depression were first eroded and then, finally, revoked — paving the way for the Great Recession, a mere seven years later.

    Three events were largely responsible for precipitating the Church Committee investigation, two notorious, the third almost lost to history.

    The first was publication in the New York Times of the Pentagon Papers in June 1971 (thanks to whistleblower Daniel Ellsberg), which showed that the government had lied to the American people about its prosecution of the war in Vietnam, in which Australia was a co-combatant. The federal government tried to stop publication of the Pentagon Paper via court order, but failed.

    The second was the Watergate break-in in June 1972, which ultimately felled a president but also revealed the use of present and former intelligence operatives in illegal activities for explicitly political purposes, which is illegal under all the intelligence services’ charters.

    The third took place before either of these two, in March 1971, and also involved a break-in, at the small FBI office in Media, Pennsylvania. The burglars were ordinary citizens (suburban parents, university professors) who called themselves the Citizens Commission to Investigate the FBI. They suspected that groups that were trying to make a better world–civil rights activists, anti-war activists–were being spied on by the FBI. They stole about 1,000 documents, which after examination they passed on to various news organisations, some of which published them, while others did not. They were never caught.

    What the Citizens Commission exposed was COINTELPRO, the FBI’s covert, sometimes illegal, program of spying on, infiltrating and disrupting domestic political organisations involved in civil rights, social justice and anti-war activities. The role of the NSA in these activities (Project Minaret), was targeting the personal communications of leaders of these organisations (i.e., Martin Luther King), legislators vocally opposed to the Vietnam War (including Church himself), as well as athletes (Muhammad Ali) and journalists.

    The intelligence agencies justified the program on grounds of national security, keeping public order and ensuring public safety. They still do. The question is whether the citizens of a democracy have the right to actively oppose the policies of elected governments–to organise and act in the legal ways available to them–without being spied upon and interfered with by that government.

    Remember, COINTELPRO and Project Minaret were targeting organisations opposed to war, oppression (racial segregation) and blatant political, economic and social inequality.
    Australians mobilised their own actions in these areas during the 1970s and 1980s. Do we know anything at all about the Government’s activities in relation to these movements, or in relation to contemporary activism concerning environmental issues and climate change? I suggest we know very little in actual fact.

    In short, Australia only looks good relative to the USA, and the USA only looks bad because citizens have been willing to risk everything to bring revelations of abuse to public attention.

    Generally speaking, Australians are rarely concerned about what their governments get up to, except when it affects them directly (taxes, health, education). We trust them to get on with the job, so we can get on with our lives. We are happy not to know, to let our political discourse be an endless game of back-and-forth without real debate on serious issues, much less real evidence (as against scraps of he-said/she-said) to inform such debates.

    Could it be because there is nothing in the Australian mode of education or our Constitution that says we — as citizens — have certain inalienable rights, and we have a duty to ensure they are not abrogated. Hmmm.

  4. Neutral

    Various organisations are now in the process of publishing guides on how to secure your online privacy.

    Basically it means using a DNS leak proof VPN, a browser with webRTC turned off and an offshore email account.

  5. The Pav

    There are three simple tests to apply.

    1)Is it a reduction of civil liberty?
    2)Will it guarantee a quantifiable improvement in security?
    3) Can it be guaranteed that the authorities will not abuse the power?

    The answers are Yes, No and No.

    This means the proposed legislation fails and should not be proceeded with.

    The Govt will proceed with it as it passes the sole criteria that the Govt and Abbott consider important.

    Does it help the Govt look good and give a bounce in the polls?

  6. AR

    Surely the clincher is about trust – do you really believe the current crop of shysters, ideologues, morons & creeps can be trusted?
    Me neither.

  7. Neutral

    Further to David Vaile’s excellent input, here’s an anonymity test you can take which shows just how much “stuff” can be collected from your browser to form your personal browsing fingerprint:

  8. Jabberwock

    If metadata is largely useless for criminal and terror cases, why are there such desperate efforts to save it? These people are not entirely stupid, so what is going on?

  9. Venise Alstergren

    If I was buying a chair I would want to sit in it.

    Yet I’m being forced to buy a ruinously expensive security measure without having any proof the bloody thing works.

    Where is my Bill of Rights declaration?

  10. esf rdggrd

    “does not appear to include download volumes”

    clearly not true, under 187A(5)(c )

    The type of a communication or of a relevant service used in connection with a communication

    (c) the features of the relevant service that were, or would have been, used by or enabled for the communication.
    Examples: Call waiting, call forwarding, [b]data volume usage.[/b]
    Note: This item will only apply to the service provider operating the relevant service: see paragraph 187A(4)(c).

Share this article with a friend

Just fill out the fields below and we'll send your friend a link to this article along with a message from you.

Your details

Your friend's details