Tuesday, February 26, 2013

Weekly aggregation

Essential is acting a little strangely for a rolling poll. The last six weeks have seen Essential report the two-party preferred vote estimate for Labor as 46 - 46 - 46 - 45 - 46 - 44. When I break this down into independent fortnightly polls I get one series that reads 46 - 46 - 46 (a flat line). The other series reads 46 - 45 - 44 (a consistent decline). This week's aggregation is informed by the second series. (This anomaly makes week-to-week comparisons of the aggregated poll a little problematic).

Newspoll has moved one point in Labor's favour since the poll three weeks ago. The lines in the next chart plot a Henderson moving average.

Comparing pollsters using a localised regression we have ...

And the aggregation: Labor's recent decline may have found a floor. Also worth noting that the national seats model has the Greens winning Melbourne. I have left the axes unchanged for the Monte Carlo seats model, but this is becoming problematic.

I am expecting another Morgan poll to be published tomorrow.

Sunday, February 24, 2013

Yet another look at the 2010 Election

I have been thinking on how to calibrate the Bayesian aggregation to get a better understanding of the actual national two-party preferred (TPP) voting intention. At the moment, the model assumes the bias across all houses sums to zero. The model needs a constraint of some kind to work. My plan was to look at the house biases for past three or four elections and to plug a multi-election average of the biases into the model as the constraint. (Unfortunately I have been diverted from that task, and this blog post explains why).

I was also thinking about what to do with Essential, which appears under dispersed compared with what you would expect from statistical theory. Furthermore, its house effect over time is inconsistent with the effects from the other polling houses. Essential's house effect is much more variable relative to other pollsters over the long run. I would not be surprised if a lot of the behaviour we see with the Essential poll is a product of its two-stage sample frame.

Anyway, I decided to run the anchored Bayesian model for the 2010 election without Essential. The results (using 1 million iterations) were as follows. (I'd ask that you excuse the indulgence of two decimal places on the house effects chart, I know the last decimal place is mostly noise).

These results surprised me. They differed substantially to what I had seen before (replicated below with a 1,000,000 simulation run).

I asked myself, what is going on here? It was time to revisit the raw data. It was a close election and much closer than most pollsters suggested (with the final Newspoll and Essential getting the closest to predicting a hung parliament).

The data from the polling houses suggested very different election campaign stories. Nielsen and Galaxy paint the picture of a campaign that did not change much. The parties finished the campaign where they the started, albeit after dipping a bit in the middle.  The Essential story is one of the Coalition consistently closing the gap. Newspoll and Morgan phone also have a gap closing story, but with Labor recovering a bit before the election. Morgan phone sees the Labor recovery sustain, but Newspoll (like Essential) saw a further decline in the final week. I am not sure what to make of the Morgan F2F narrative.

These narratives can be highlighted with a short-run LOESS regression for each house.

There are a number possibilities that might explain the inconsistencies and wrinkles in and between the above charts.

There may have been a further decline in Labor's vote share in the last two days of the campaign that was not picked up by the polls. Personally, I am not convinced by this. 

I suspect the first Newspoll reading in the period was atypically high (compared with where Newspoll typically sits in house effect terms). Which means, I suspect the election was pretty close from at least month out (but before that it was more favourable for Labor with the Gillard honeymoon effect - discussed further below).

Another possibility, which I have still to explore, is that the TPP vote estimates based on preference flows in 2007 were unrealistic. 2007 was on of those turning point elections where there was a clear mood for change.  It may be that preference flows in 2010 were unusual, and my reliance on them for predicting 2013 is problematic.

A confounding factor is that Julia Gillard only replaced Kevin Rudd as Prime Minister on 24 June (less than a month before the election was announced on 17 July). The election was held in the post Gillard honeymoon period, and this may have affected some polling houses more than others. If we take a slightly longer time frame we can see the following, where it would appear that Essential and Morgan F2F were the most consistently honeymoon affected pollsters (although the Morgan phone poll had a bit of a blip there).

There is much to think about here.

Monday, February 18, 2013

One of these things is not like the others

Recall that Sesame Street song ...

Well something like that is happening with the opinion polling at the moment. One of these pollsters is not like the others.

Once again I am left pondering Essential's steadfastness; when the other houses have moved away from Labor by a comparable 4 or 5 percentage points. It will be interesting to see if this divergence is just a point in time thing (either the other three move back to where they were or Essential moves), or whether it turns out to be a longer-term discontinuity. 

The weekly aggregation is unchanged on last week (46.1 to 53.9). But I am less comfortable with the notion that the zero-sum across polling houses is where the actual national voting intention lies. I suspect it is 0.2 or 0.3 percentage points to the left of the current average. I am working on strategies to better quantify when the sum-to-zero constraint may differ materially from the national voting intention (as it did at the last election).

Explanatory notes on the Bayesian aggregation can be found on the Technicals page.

Wednesday, February 13, 2013

Updated poll aggregation

Last week we had a surprise from Newspoll, when it moved five percentage points towards the Coalition. Now Morgan has released a weekly poll for the last weekend that has moved four percentage points over the previous weekend; also to the Coalition. This movement from two polling houses has influenced the poll aggregation in recent weeks.

Explanatory notes on the Bayesian aggregation can be found on the Technicals page.

Tuesday, February 12, 2013

Weekly poll aggregation

There was only one new poll since last week's. It was the Essential rolling poll, which combines two weekly polls. It moved this week, from 46 to 45 per cent Labor two-party preferred.

Saturday, February 9, 2013

A different look at house effects over time ...

House effects are the systemic biases that consistently affect a pollster's published estimate of the population voting intention. In the Bayesian model, we assume they a constant over time.

In this analysis, using localised regressions (LOESS), we can see the slow drift in Essential's population estimate compared with the other three polling houses (that poll on a fairly regular basis). This is not the first time we have reflected on Essential's drift. But it is the first time we have done it with LOESS.

Another factor I have been thinking about is non-linearity in the response of polling houses to changes in the population voting intention. If you consider the 365-day-LOESS-span chart, it can be argued that over the past 18 months Morgan's face to face poll has been more responsive to shifts in public voting intention for Labor than Essential (and perhaps Newspoll). What is less clear to me is whether some polling houses are over-responsive and/or others are under-responsive.

Some key dates to think about in considering this period include:
  • Abbott replaced Turnbull as Opposition Leader - 1 December 2009
  • Rudd announced that the Government would delay implementing an emissions trading scheme - 4 May 2009
  • Gillard replaced Rudd as Prime Minister - 24 June 2010
  • 2010 election - 21 August 2010

And the combined LOESS charts for the same periods ... the drift in Essential's house effect means that the combined LOESS leans more towards Labor at the beginning of each chart than it does at the end of that chart.

Tuesday, February 5, 2013

Weekly poll aggregation

This week we have two and a half new opinion polls. One of them - Newspoll - lurched dramatically to the Coalition. Three weeks ago, the Newspoll two-party-preferred estimate was 51 to 49 in the Coalition's favour. Last weekend, the Newspoll estimate was 56 to 44 in the Coalition's favour. A five percentage point movement.

Last weekend's Galaxy poll was pretty much unchanged from the beginning of December (which is consistent with the aggregated poll as it was until last week). For the rolling fortnightly poll from Essential (half of which was sampled last weekend and the other half during the previous weekend) we also have a steady-as-she-goes poll result. All of which leaves me wondering, is the Newspoll result a rogue poll?

While "rogue" (at least in part) would be my hunch, there is currently no way of knowing for sure. We will need to see a few more polls before we can form an informed judgement on this particular poll. Nonetheless, such a dramatic movement in this Newspoll has moved the aggregated poll result (albeit less so).

Update 6 February

Morgan has broken with its usual practice and released a weekly poll: 50.5 to 49.5 in the Coalition's favour. This was a one-percentage point move to the Coalition over the previous fortnightly poll. Dropping these numbers into the Bayesian model sees an unchanged endpoint:

Update 8 February

I have been asked about the latest LOESS aggregation. At 47.1 per cent, it yields a very similar result to the Bayesian aggregation.