Confessions of a failed forecaster

Let's be frank: I had a shocking election. My aggregated forecast of 51.4 per cent for the Coalition in two-party preferred (TPP) terms was off, way off. The final outcome was a good percentage point lower at 50.35 per cent. 

My error was simple. I had assumed that the pollster biases from the 2013 election would be much the same in 2016. Ironically, I was alerted to the error of my ways shortly before the election when Nate Silver blogged in early June 2016, 

The good news is that, over the long run, the polls haven’t had much of an overall bias, having underrated Republicans in some elections and Democrats in others. But the bias has shifted around somewhat unpredictably from election to election. You should be wary of claims that the polls are bound to be biased in the same direction that they were two years ago or four years ago.

Although I run both 2013-biased and unbiased models, I doubled down on the middle ranked biased model without testing the robustness of my assumptions. To add insult to injury, my unbiased models came in much closer to the final result. The unbiased TPP model projected the Coalition would get 50.6 per cent of the TPP vote share. The unbiased primary vote model projected 50.5 per cent.

Now that I have got that off my chest, let's have a closer look at the path of the polls over the election period and evaluate the performance of the pollsters. If we anchor a daily walk of the TPP estimate from aggregating the polls to the election outcome, we get the following.

My results suggest that Ipsos was the most accurate pollster over the election period. But to be fair, There was only a whisker in it between Ipsos, Galaxy, ReachTEL and Newspoll.

These results also suggest that Turnbull snatched outright victory (albeit a slim one) from what looked like a hung parliament early in the longer-than-usual election period. The cloud of poll results in the last quarter of this period were the most favourable to Turnbull.

If we focus on the primary votes, the story becomes a little more nuanced. Over the course of the election period the majors - but particularly Labor - lost primary vote share to the Others. This movement from Labor to Others may have been behind the Coalition's whisper thin win in the end.

While the Greens improved on their 2013 result (from 8.65 to 10.23 per cent of the primary vote share), they under-performed against their pre-election polling. The Green's primary vote trajectory was downwards for most of the period

The Coalition did worse than 2013 with preference flows. If they had achieved the same preference flows in 2016 as 2013, they would have achieved a TPP vote share of 50.6 per cent. This is significant: Up until the 2016 election, it was possible that the historically low preference flows in 2013 where anomalous.

My suspicion is that we are seeing fracturing of he left in Australian politics, where more and more people who do not want to vote Coalition, also feel uncomfortable with Labor in the first instance (even though in most cases the preference from those voters ultimately flows to Labor).

Well that is enough confession for this Sunday. Over the coming months I will look at other aspects of the 2016 election in more detail. As polls this far out from an election are of little predictive value, I will take a holiday from poll aggregation for the next 12 to 18 months or so.