By now, the chorus of indignation in response to the government’s proposed rules around the collection of metadata is well established. But is this reaction warranted? On balance, it is not. While we must always be vigilant about our personal freedoms and potential government encroachment, this latest episode of Big Brother hand-wringing has been overblown and heedless of some basic realities of the digital lives most of us live.

The fact is that metadata is everywhere. We shed it like digital dander containing our personal DNA as we go online, use our smartphones, and send and receive emails. Every digital step we take — and for most people that is countless steps every day — creates it.

This digital dander is hoovered up by telecommunications companies, internet service providers and large offshore multinationals. After the multinationals ship it offshore, unidentified, unregulated marketers and engineers use it in unfettered ways for who-knows-what purposes (usually to market goods to us). There is no audit trail, no accountability and absolutely no oversight by any government.

For the most part, this hoovering gets only minimal attention. There was some press when it became publicly known that Apple iPhones were tracking every step users made and a bit more muted commentary when Google changed its privacy policy to allow the tech giant to aggregate data across its 40+ “free” services. An Android smartphone user who accesses Gmail has his data cross-referenced with Google maps, Google searches and so on and so forth. The creation of an ever more detailed data file on each individual user — arguably the most comprehensive data set held by a private company about private citizens in history — elicited scarcely a whimper from civil liberties groups.

“It is the sheer breadth and audacity of the private sector’s collection of data that should give us pause before we rush to the barricades in protest of the federal government’s proposal.”

Still, it can be argued that there is a difference between a private company taking your data and a government doing so, as the government has the power to charge, compel and prosecute. Fair enough. In the current case, is the government justified in wanting this data? The answer hinges on law enforcement. Just as our lives have moved online, so has crime and a range of illegal activity. Solving many serious and heinous crimes now revolve around digital evidence, often located based on the patchwork quilt of metadata.

It is the sheer breadth and audacity of the private sector’s collection of data that should give us pause before we rush to the barricades in protest of the federal government’s far more modest proposal. According to what we’ve learned, the federal government plans to compel Australian telecommunications companies and ISPs to retain a small subset of metadata for a period of two years so that police agencies and ASIO, if the need arises, can apply for those records in order to investigate serious criminal offences and attacks on Australian society.

But what is this metadata? Is it the sort of personal material gobbled up by private sector players? Hardly. Communications Minister Malcolm Turnbull has given the most clarity to this debate we’ve seen in the past four years from either side of government (remember, it was Labor that first proposed this, so it’s bi-partisan): the government is seeking phone call records be retained (telcos generate these for billing purposes anyway), as well as our dynamically allocated IP numbers when we go online. These numbers are assigned to us each time to use the internet and are required to track back to a user at the time. On any day several people may be allocated the same number, so it is critical, to protect the rights of innocent people, that the right subscriber is tied to a possible offence. It is, in essence, a living white pages phone book, retained for two years, in case a piece of our technology is implicated in a crime. Just to emphasise the point: metadata need not be only incriminatory, it can be exculpatory as well, meaning that having a system in place that can properly collect and assess this data serves all our interests.

That said, we must ensure that the system won’t be abused. First, the number of agencies who can access this type of data must drop. Second, the offences for which this kind of data can be sought needs to be narrow. And lastly, while the government has good grounds for demanding the retention of such information for two years, it should also mandate its destruction at that time unless there is a genuinely sound reason for keeping it.

The future is likely to produce a lot more digital dander, and only considered and prudent planning will make that future one in which the right to privacy and the public good are balanced.