Has Resolve Strategic changed its polling methodology?

The aggregation models I use make many simplifying assumptions. One assumption is that the polling methodology adopted by each polling firm remains unchanged over time. If a pollster indicates that they have changed their polling methodology, then I would treat the subsequent polls as a different series to the prior polls. I have done this with Essential when they published a change to their polling methodology.

When I look at the recent polls from Resolve Strategic, it looks like the polls from 2024 are less favourable to Labour than the polls from the prior two years. In the first chart below, we can see that the last poll result was almost four percentage points more favourable to the Coalition than the long term average for Resolve Strategic. In the second chart below, we can see that historically Resolve Strategic has been the most favourable poll for Labor (on average). And in the third chart below, we can see that the individual Resolve Strategic polls in 2022 and 2023 (indicated with the letter h) were often (but not always) well above the aggregation, and two of the three polls in 2024 are not. 

Now, this could be just the usual random noise and chance that comes with statistics. And in coming months we will see a return of the prior patterns of systemic house bias at Resolve Strategic. But this apparent change could also be the result of a change in polling methodology at Resolve Strategic. At the moment I don't know which explanation is the most plausible. While I have looked, I have not seen a statement on any methodology change from Resolve Strategic (please provide a link below in the comments if I have simply missed it).

If this apparent change in house bias is sustained in the next couple of Resolve Strategic polls, I will assume that there has been something of a change in polling methodology, and I will separate the 2024 and subsequent polls into their own series. 

Please note: this is not a criticism of Resolve Strategic. I have enormous respect for all pollsters, and I appreciate that opinion polling is much harder today than it was (say) 35 years ago (when almost every house had a landline and no-one had mobile phones). At the moment, I am just observing that the last three polls from Resolve Strategic perhaps look a little different from the earlier polls, and I am musing why this might be the case. Nonetheless, I would be somewhat disappointed if it turns out that Resolve Strategic has changed its polling methodology but has not communicated how it has changed and why it has changed.

The polls are getting close

I have updated my poll aggregations, and it is getting close. The Gaussian Random Walk (GRW) model has Labor on 50.1 per cent of the two-party preferred (2pp) vote share if an election was held now. The less tested Gaussian Process (GP) model has Labor on 49.7 per cent.

My simple local regression models have the 2pp vote share at 50 per cent even. 

Nonetheless, there remains a substantial primary vote for the non major parties, which makes it harder to project who will form government on the basis of the 2pp vote share.

Comments

At this point, my best guess is that the polls are currently pointing to a minority Labor government, with an increased cross bench. 

Part of the reason I have Labor ahead when the 2pp polling aggregte is around 50/50 is historical. First term governments usually win a second term. There is an inertia built into the super-majorities governments typically get in their first term (and with the Herculean task faced by oppositions to rebuild after change-of-government elections). The last one-term Federal government was the Scullin government that had the misfortune of being elected just after the 1929 Wall Street Crash, in the midst of the Great Depression.

A second reason is that the 2pp poll aggregates have only just hit 50/50. I would want to see the 2pp poll aggregates consistently below 50 per cent for some months before seriously reconsidering Labor's chances at the next election. Governments have the advantage going into an election. They have the advantage in setting the date for an election, and they can use the expertise of the public service to better prepare for an election (before it has been called). 

A third reason for caution is the possibility that the latest Resolve Strategic poll is a statistical outlier. That 2pp poll came in at 50/50, but in the aggregation models, given historical systemic house bias patterns, the latest poll was treated more like 53.5 for the Coalition and 46.5 for Labor. It is possible that Resolve Strategic has changed its polling methodology, and we will therefore need to change how we treat these Resolve Strategic polls in the future. Ot it could just be part of the usual noise of opinion polling, and going forward we will see the long-standing systemic bias patterns return to how they have been.

Finally, it is worth noting that getting less than 50 per cent of the 2pp vote does not necessarily mean defeat. Governments have won in the past with narrow 2pp vote losses. For example: Howard in 1998 with 49.0 per cent, Hawke in 1990 (49.9%), Gorton in 1969  (49.8%), and Menzies in 1961 (49.5%) and 1954 (49.3%). 

Various Bayesian models for modelling voting intention

I run a number of Bayesian models of voting intention for the next Australian Federal election. 

The first is a Gaussian Random Walk (GRW) which models the voting intention by assuming (1) voting intention on any single day only varies a little from the day before, (2) the polls are a noisy indication of population voting intention, and (3) individually pollsters have systemic biases but collectively across all pollsters the polls are unbiased. This is the same model I have used in past elections. It is also my preferred model. At the moment it shows Labor two-party preferred (2pp) voting intention at 50.7 per cent.

The second model I run - a Gaussian Random Walk with a Left Anchor (GRWLA) - augments the first model by anchoring the hidden voting intention to the result at the last election. By imposing a left anchor, I am assuming some things: First, that any honeymoon effect after the election is consistent with my assumption of minimal voting intention change from day to day. Second, that pollsters retain the same methodological approach from the time of the last election. I am not convinced by either of these assumptions. At the moment it has Labor's 2pp voting intention at 49.2 per cent.

The third model I run is a Gaussian Process (GP) model. Rather than model hidden voting intention on every day, it only models hidden voting intention on polling days. The model assumes that the hidden voting intention for polls that are nearby in time will be more correlated than for polls that are further away in time. This relationship between polls based on their temporal distance from each other is expressed mathematically in the model using an exponentiated quadratic kernel. Like the above models, this model also makes adjustments for systemic pollster biases. At the moment it has Labor's 2pp voting intention at 50.3 per cent.

I am still exploring this approach, however, it is sensitive to where the "mean" across the data points is set (with a strong tendency to mean reversion at each end of the series, and for periods through the series where there are few polls). I have set the mean to the average of the last 10 polls. 

We can compare the central tendencies of these approaches as follows. You will note the "mean reversion" evident in the early part of the GP series.

These models are encoded in Python and they can be seen on my github repository. If you want to look at the models closely, you should look at bayes_tools.py and the notebook _poll_agg.ipynb

The GRW and GRWLA models are based on the work of Simon Jackman in Bayesian Analysis for Social Sciences (2009).

Mid April 2024 Polling Update