TPP voting intention movements and polling house dispersion

I have been thinking about the two questions Kevin Bonham posed in his comments. While my answers are not perfect, I think they go some way to answering Kevin's questions. But I wont take them in the order Kevin asked. I will deal with the easier question first.

It related to the average level of week to week movement in population voting intention. I have taken a 31-day (ie. one month) Henderson moving average from the Bayesian aggregation as a proxy for underlying voting intention. Looking at the absolute movement for every possible seven day set in the 6 month period under analysis we find:

• The average weekly movement (in absolute value terms) was 0.25 percentage points.
• The biggest weekly movement over the period was 0.72 percentage points.
• The smallest weekly movement was 0.01 percentage points (ie. pretty much unchanged)

If anything, I expect the movements we have seen over the last six months are atypical. I would expect a longer run analysis would yield a smaller average weekly movement.

The chart that goes with this analysis follows. The fine line is the median of the samples from the Bayesian aggregation. The thicker line in the 31-day Henderson moving average, The major breaks on the x-axis are calendar months and the minor breaks are weeks:

Note: I also looked at a 61-day moving average (on the possibility that population voting intentions might be more sticky), but this did not affect my findings substantially.

Kevin's second question related to checking for over-dispersion in the new Morgan multi-mode poll. I started thinking about this question by plotting a chart of bias-adjusted polls (using the house effects from the Bayesian aggregation). This chart (the third below) also includes 95% confidence intervals (1.96 times the standard error on the proportional vote shares for Labor).

Interestingly, the Bayesian aggregated Labor TPP vote estimate is within the 95% confidence interval for all but one of the bias adjusted polls (two polls prior to the max line on the third chart). This result was a touch better than I had expected from theory (but not implausibly so). There are 50 published polls in our analytical set; if 5 per cent were to lie outside of the 95% confidence interval, then two or three polls would be the most likely number of bias adjusted polls not to include our estimated population parameter in their confidence interval.

The relevant charts follow. The first chart is from the Bayesian aggregation and includes the raw polling results. The second chart is of the estimated house effects (which I used to create the third chart). The third chart is of the bias-adjusted polling results (ie. the raw polling results minus their house effect) with associated confidence intervals:

This chart got me thinking about the normal distribution. From the normal distribution we know that 68.27 per cent of cases should lie within the first standard deviation from the mean. We know that 31.46 per cent of cases should lie between the first and second standard deviations (in both directions). Furthermore, we know that 4.55 per cent should lie beyond two standard deviations from the mean.

While not a foolproof mechanism for testing for over-dispersion, we can look at the distributions of  bias-adjusted polling results for each polling house to see how they relate to these theoretical expectations of the normal distribution and the 31-day HMA proxy of population voting intention. The preliminary results follow (and these can be compared against an expected distributional outcome of 68-31-5 as just discussed):

• Essential (N=11) was distributed 91-9-0 suggesting under-dispersion (ie. in 91 per cent of cases the 31-day HMA was within one standard deviation of the Essential bias-adjusted poll result)
• Galaxy (N=6) was distributed 100-0-0 suggesting under-dispersion
• Morgan face-to-face (N=9) was distributed 56-33-11 suggesting normal to slightly over-dispersed
• Morgan multi (N=9) was distributed 33-67-0 suggesting over-dispersion
• Newspoll (N=10) was distributed 50-50-0 suggesting over-dispersion
• Nielsen (N=5) was distributed 100-0-0 suggesting under-dispersion

While this test aligned with my perceptions, obviously some questions need to asked. For example, can we substitute a mean for the median from the Bayesian aggregation? Does the central limit theorem apply with such small samples? Does it allow us to assume that sample distributions will be normal (regardless of the distribution of the entire population)? Are the results an artifact of the aggregation, rather than a characteristic of the polling houses? How are the results affected by house rounding? All good questions to ponder further.