What started out as an exercise in looking at whether we can see any structural effects of incumbency on the results of “better party/leader to manage” polls over time – whether the act of being in government itself has an impact on these metrics – quickly became something else entirely as the data spoke in what were slightly unexpected ways.
If a party or political leader increases their standing on any particular issue – say on “better party/leader to manage the economy” – what also generally happens is that they increase their standings on *all* issues. Not only do the results of “best party/leader to manage” questions all move together through time, but they’re also tied up in massive correlation with generic approval ratings and Better Prime Minister ratings, at least to the point where any given poll result on these better party to manage questions doesn’t have anything particularly useful to say.
To show how it all works, we’ll use the Newspoll results through time for four issues – two issues that are usually considered Labor strengths (Health & Medicare and Education) and two issues that are usually considered Coalition strengths (National Security and the Economy).
On the actual question front, until recently Newspoll used to ask two separate questions here. The first question – and the question they still run –asks:
Which one of the following (ALP, Liberal and National Party Coalition, or someone else) do you think would best handle the issue of: (insert issue)
The other question, and one they haven’t asked since September 2008, goes:
Which of (Leader) do you think is more capable of handling (insert issue)
Historically, these different questions were asked about a month apart at various times over the last 5 years. That allows us to compare the results of each question on each issue for reasons that will become apparent a little later on.
First up, the raw results of both the best leader and the best party to handle each of the four issues.
The two things that stand out are firstly, the way that the leader and party results for each party on a given issue move together, with the better leader results being a little more volatile. That’s not particularly surprising.
The second thing that stands out is how each pair of issues that a party is considered strong on (Health & Medicare and Education for Labor, National Security and Economy for the Coalition) not only move together, but produce almost identical results at both the party and leader level. It becomes more obvious when we chart them against each other – party results first, followed by leader results.
As a party or leader increases or decreases their position on Health and Medicare, there is an almost identical movement for them on Education. Similarly, as a party or leader increases or decreases their position on National Security, there is an almost identical movement for them on the Economy.
In fact, the results from all of these issues move together nearly all of the time.
If we take the first difference of these poll series, where we measure the change in value for each observation (for example, if Education was on 40 in one poll and came in on 50 the next, the value for this observation would be +10), we can track the way these changes all move together on all issues.
Remember, these results also contain polling noise which is likely to be washing out the correlation a little!
The other thing we can do is plot the change in each series for the Party question against the change in each series for the Leader question. Coalition results are in blue, Labor results are in red.
Not only are leadership effects tightly correlated with party effects, but all the issues behave in a way that suggests they too are tightly bound up with not only the leadership/party correlation, but with correlation between each other.
If the results of these polls all move in the same direction approximately all of the time – what actual, meaningful information can we extract out of them?
For instance, if Labor increases their lead in “better party to handle Education”, what does that actually mean when the issues are all moving together? What does it mean when the leadership of each party is entangled with the results?
At this stage, I was thinking that generic approval must be playing a major part in these results – something has to be, and that was the most obvious place to look. So, time for some regression work (stop groaning – I heard that!)
If we build two new series, firstly the change in satisfaction rating with each leader and secondly, the change in preferred Prime Minister for each leader – that gives us a couple of good proxies for generic approval. So, for instance, if the Preferred PM for the Labor leader increased by4 points between June/July 2005 and October/November 2005 (the first two observation periods), the value for this observation would be +4
If we regress the “Best Party/Leader to manage” results against these generic approval ratings for these four issues, this is what we get:
For the non stats types:
The dependent variable is the thing we are trying to explain.
The independent variable is the thing we are trying to explain “changes in the dependent variable” with.
The Coefficient Value tells us how much change we would expect to see in the value of the dependent variable if the independent variable changed by 1 point.
The R-sqr value tells us how much of the variation in the dependent variable is explained by changes in the value of the independent variable.
The Standard Error, T-stat and p value are for the nerdherd. Suffice to say that these are all strongly statistically significant relationships.
For instance, 60.8% of the change in “Best Party to handle” any of these four issues can be explained by changes in the preferred PM value of the leader of that party. For every 10 points that the party leader increases his preferred PM value, the better party to manage result increases, on average, by around 3 points.
Changes in satisfaction ratings or preferred PM ratings separately explain between 20% and 78% of the variation we see in the better party/leader poll results. Combined, they would explain more of the variation, but there is so much correlation between satisfaction and preferred PM it’s virtually impossible to pull their joint effect on these issue polls out of the data.
So, if:
– all the issues move together approximately all of the time
– between 20% and 78% *at least* of the variation in issue polls are a function of generic approval levels of the party and its leader
– changes in those issues that are considered strong by a party are almost identical to other issues considered strong by that party
…what meaningful value can we actually extract from them? We know what we cannot extract meaningfully and that’s guff like this.
But what can we extract?
Perhaps long term relative changes in which party is best perceived to manage a given issue, perhaps we can identify if issues cease to become a strong issue for a party over a long period of time. Any sharp jump in value for a given issue that is above and beyond that achieved by other issues in that poll is also something that would be meaningful and worth taking a second look at.
But for ordinary poll to poll movement, we can’t actually pull much pointy end value out at all because large parts of the variation are simply a function of generic approval of the party leaders.
These questions could be analysed using respondent level data and there would be value to be had there (which is what the internal party polling does) – but these raw aggregations are a little blasé in the broader scheme of things and should be treated with the utmost caution.
On another related issue, Newspoll asks the following question on issue importance:
Thinking about federal politics, would you say each of the following issues is very important, fairly important or not important on how you personally would vote in a federal election?
It’s a real shame that they only publish results for the “very important” response. The “fairly important” or “not important” results would be a great addition to understanding how the intensity of political issues is changing over time.
Leave a comment