In the wake of an election result that defied nearly every item of conventional wisdom concerning the game of British politics, players in Australia will be alert — perhaps over-alert — to lessons that might apply here.

In particular, the Prime Minister — who, by the account of Laurie Oakes, is eyeing off a double dissolution after this week’s budget — would have found much to warm his Anglophile heart as the Conservative government’s hawkish austerity policies found unexpectedly strong support among the English middle class, despite having long weighed it down in opinion polls.

Since Abbott’s bullishness about his electoral prospects is reportedly based on a conviction that his underlying position is two points stronger than whatever the polls might be showing, it would also not have escaped his notice that the result was apparently swung by many Tories who had shown themselves to be “shy”.

That, at least, is the most widely offered theory for the comprehensive failure of the opinion polls to accurately predict the result.

The unanimity of the pollsters on election eve had been quite remarkable, with 10 out of 11 agreeing there was not more than a single percentage point in it — the sole exception being a poll that found Labour two points in front. In the event, the Conservative vote across Great Britain (thus excluding the special case of Northern Ireland) came in about 4% higher than the poll average at 37.8%, while Labour’s landed about two points short at 31.2%.

British political folklore already had to hand a ready explanation for such an anomaly, since the one comparable failure of modern times had also involved the unexpectedly decisive return of a Conservative government.

In 1992, John Major’s Conservatives outpolled Labour by 7.5% after late campaign polls had uniformly pointed to a tight race, in doing so securing one last term for an administration that first came to power under the leadership of Margaret Thatcher in 1979.

It was widely hypothesised at the time that those who planned on voting Conservative had been hesitant to say so — which, according to taste, could be put down to suffocating conventions of political correctness that inhibited people from speaking their minds, or an understandable reluctance to identify with the party’s grasping ideology.

Pollsters took such explanations seriously enough to revise their methodologies to account for possible Labour biases, most commonly by inquiring into past voting behaviour or party identification so that responses could be weighted to align with the previous election result.

However, there are reasons to doubt whether the problem this time around was an outbreak of shy Toryism on sufficient scale to overwhelm measures already in place to account for it.

For one thing, the direction and the scale of the error was consistent across all methods of polling, including online ones in which respondents faced no live interviewer to feel shy towards.

The “shy Tory” thesis might also ring false to Australian observers in light of the most recent failure of polling in this country, when Labor’s two-party vote at the Queensland state election on January 31 was underestimated by around 3%.

Of course, close observers of Australian polling will be aware that the specific problem on that occasion was a misallocation of minor party and independent preferences, an issue that wouldn’t arise in Britain’s first-past-the-post voting system.

Using the standard by which their British counterparts would be judged, namely the primary vote, the Queensland election found the main Australian pollsters maintaining a long-standing record of accuracy that would be the envy of any firm operating in Britain.

Since pollsters in the two countries aren’t doing anything radically different from each other, the disparity very likely arises from differing circumstances in the two countries, of which the most obvious is compulsory voting.

Australian experience suggests respondents are, in fact, surprisingly honest in identifying which party they favour — at least when they can be pinned down for an answer, something that is increasingly difficult to achieve in an era of high and ever-increasing non-response rates.

The difficulties arise with secondary questions for which pollsters need accurate answers if they are to correctly model the result.

In the Australian context, it appears many respondents who favour minor parties are unable to accurately account for what they will do with their preferences. This causes pollsters to favour allocating preferences in line with the results of the previous election, a method that can be found seriously wanting when a significant realignment occurs.

In Britain, pollsters ask respondents how likely they are to vote, only to find that many who talk a good game about their devotion to civic duty end up finding better things to do when election day rolls around.

On this particular occasion, pollsters appear to have produced over-optimistic turnout estimates — except in Scotland, where the remarkable result had in fact been accurately foreseen.

In hindsight, it makes perfect sense that the burst of democratic enthusiasm unleashed by the independence referendum should have carried over to the general election, to the very great advantage of the Scottish National Party.

But south of the border, it seems that potential Labour voters especially were not sufficiently enthused about the product on offer to make the necessary effort.

Another telling anecdote about the failure of British polling concerns a landlines-plus-mobiles survey conducted on the eve of the poll by the newish firm Survation, which did in fact precisely nail the result.

That, at least, is what the firm’s CEO tells us, and we shall have to take his word for it, because the peculiarity of the result caused him to shy away from publishing it.

For an indication as to why a pollster might behave in this fashion, consider the example of Australia’s very own Roy Morgan Research. It lost its contract with The Bulletin after going out on a limb in predicting a Labor landslide in 2001, and has never been taken entirely seriously by the media since.

When the day does arrive that Australian pollsters do collectively embarrass themselves, there’s a strong chance that it will do so as a result of “herding” effects, in which second-tier pollsters tweak their methodology if results fail to conform with a consensus view established by recognised industry leaders.

Finally, it’s worth considering that there may have been a dynamic at work that would have eluded even the sharpest of polling methodologies, namely a dramatic break among late deciders in favour of the Conservatives.

In other words, it may simply have been that cautious voters faced by ongoing economic uncertainty decided to opt for the devil they knew — offering further encouragement, if any were needed, to Tony Abbott.

Peter Fray

Get your first 12 weeks of Crikey for $12.

Without subscribers, Crikey can’t do what it does. Fortunately, our support base is growing.

Every day, Crikey aims to bring new and challenging insights into politics, business, national affairs, media and society. We lift up the rocks that other news media largely ignore. Without your support, more of those rocks – and the secrets beneath them — will remain lodged in the dirt.

Join today and get your first 12 weeks of Crikey for just $12.


Peter Fray
Editor-in-chief of Crikey