Scott Morrison has just had his two best polls ever: Ipsos and Essential have both given him 48 per cent of the two party preferred vote. Although the Bayesian model has moved in the direction of the Coalition, it needs sustained polling at this level before it would be fully convinced that his vote share is actually 48 per cent.

Turning to the moving averages, which I use to cross-check the Bayesian model, we can see they are all over the place with the latest polls. It is the same message: You need more than two polls at a 48 per cent to be convinced this is the new normal (particularly in the context of the most recent Newspoll at 45 per cent).

The primary vote Bayesian models are next.

And the possible TPP from the primary models ...

# Mark the Ballot

Psephology by the numbers

## Tuesday, November 20, 2018

## Saturday, November 17, 2018

### Updated primary vote share model

For the 2019 election, I have explored a number of models for
aggregating the primary vote shares, including models based on Dirichlet processes and centered logits. These older models were complicated and
slow. They included constraints to ensure vote shares always summed to
100 per cent for every sample from the posterior distribution.

The model I currently run is very simple. It is based on four independent Gaussian processes for each primary vote series: Coalition, Labor, Greens and Others. In this regard it is very similar to the two-party-preferred (TPP) model.

The model has few internal constraints and it runs reasonably fast (in about 3 and half minutes). However the parameters are sampled independently, and only sum to 100 per cent in terms of the mean/median for each parameter. For the improved speed (and the absence of pesky diagnostics) this was a worthwhile compromise.

The Stan program includes a generated quantities code block in which we convert primary vote intentions to an estimated TPP vote share, based on preference flows at previous elections.

While the TPP model operates in the range 0 to 1 (with vote share values typically between 0.45 and 0.55), the new model centers all of the primary vote observations around 100. This ensures that for each party the analysis of their vote shares are well away from the edge of valid values for all model parameters. If we did not do this, the Greens vote share (often around 0.1) would be too close to the zero parameter boundary. Stan can get grumpy if it is being asked to estimate a parameter close to a boundary.

The key outputs from the new model follow. We will start with the primary vote share aggregation and an estimate of house effects for each party.

We can compare the primary vote shares for the major and minor parties.

And we can estimate the TPP vote shares for the Coalition and Labor, based on the preference flows from previous elections

The code for the latest model is as follows.

The Python program to run this Stan model follows.

The model I currently run is very simple. It is based on four independent Gaussian processes for each primary vote series: Coalition, Labor, Greens and Others. In this regard it is very similar to the two-party-preferred (TPP) model.

The model has few internal constraints and it runs reasonably fast (in about 3 and half minutes). However the parameters are sampled independently, and only sum to 100 per cent in terms of the mean/median for each parameter. For the improved speed (and the absence of pesky diagnostics) this was a worthwhile compromise.

The Stan program includes a generated quantities code block in which we convert primary vote intentions to an estimated TPP vote share, based on preference flows at previous elections.

While the TPP model operates in the range 0 to 1 (with vote share values typically between 0.45 and 0.55), the new model centers all of the primary vote observations around 100. This ensures that for each party the analysis of their vote shares are well away from the edge of valid values for all model parameters. If we did not do this, the Greens vote share (often around 0.1) would be too close to the zero parameter boundary. Stan can get grumpy if it is being asked to estimate a parameter close to a boundary.

The key outputs from the new model follow. We will start with the primary vote share aggregation and an estimate of house effects for each party.

We can compare the primary vote shares for the major and minor parties.

And we can estimate the TPP vote shares for the Coalition and Labor, based on the preference flows from previous elections

The code for the latest model is as follows.

// STAN: Primary Vote Intention Model // Essentially a set of independent Gaussian processes from day-to-day // for each party's primary vote, centered around a mean of 100 data { // data size int<lower=1> n_polls; int<lower=1> n_days; int<lower=1> n_houses; int<lower=1> n_parties; real<lower=0> pseudoSampleSigma; // Centreing factors real<lower=0> center; real centreing_factors[n_parties]; // poll data real<lower=0> centered_obs_y[n_parties, n_polls]; // poll data int<lower=1,upper=n_houses> house[n_polls]; // polling house int<lower=1,upper=n_days> poll_day[n_polls]; // day on which polling occurred //exclude final n parties from the sum-to-zero constraint for houseEffects int<lower=0> n_exclude; // period of discontinuity and subsequent increased volatility event int<lower=1,upper=n_days> discontinuity; // start with a discontinuity int<lower=1,upper=n_days> stability; // end - stability restored // day-to-day change real<lower=0> sigma; real<lower=0> sigma_volatile; // TPP preference flows vector<lower=0,upper=1>[n_parties] preference_flows_2010; vector<lower=0,upper=1>[n_parties] preference_flows_2013; vector<lower=0,upper=1>[n_parties] preference_flows_2016; } transformed data { int<lower=1> n_include = (n_houses - n_exclude); } parameters { matrix[n_days, n_parties] centre_track; matrix[n_houses, n_parties] pHouseEffects; } transformed parameters { matrix[n_houses, n_parties] houseEffects; for(p in 1:n_parties) { houseEffects[1:n_houses, p] = pHouseEffects[1:n_houses, p] - mean(pHouseEffects[1:n_include, p]); } } model{ for (p in 1:n_parties) { // -- house effects model pHouseEffects[, p] ~ normal(0, 8.0); // weakly informative PRIOR // -- temporal model - with a discontinuity followed by increased volatility centre_track[1, p] ~ normal(center, 15); // weakly informative PRIOR centre_track[2:(discontinuity-1), p] ~ normal(centre_track[1:(discontinuity-2), p], sigma); centre_track[discontinuity, p] ~ normal(center, 15); // weakly informative PRIOR centre_track[(discontinuity+1):stability, p] ~ normal(centre_track[discontinuity:(stability-1), p], sigma_volatile); centre_track[(stability+1):n_days, p] ~ normal(centre_track[stability:(n_days-1), p], sigma); // -- observational model centered_obs_y[p,] ~ normal(houseEffects[house, p] + centre_track[poll_day, p], pseudoSampleSigma); } } generated quantities { matrix[n_days, n_parties] hidden_vote_share; vector [n_days] tpp2010; vector [n_days] tpp2013; vector [n_days] tpp2016; for (p in 1:n_parties) { hidden_vote_share[,p] = centre_track[,p] - centreing_factors[p]; } // aggregated TPP estimates based on past preference flows for (d in 1:n_days){ // note matrix transpose in next three lines tpp2010[d] = sum(hidden_vote_share'[,d] .* preference_flows_2010); tpp2013[d] = sum(hidden_vote_share'[,d] .* preference_flows_2013); tpp2016[d] = sum(hidden_vote_share'[,d] .* preference_flows_2016); } }

# PYTHON: analyse primary poll data import pandas as pd import numpy as np import pystan import pickle import sys sys.path.append( '../bin' ) from stan_cache import stan_cache # --- check version information print('Python version: {}'.format(sys.version)) print('pystan version: {}'.format(pystan.__version__)) # --- curate the data for the model # key settings intermediate_data_dir = "./Intermediate/" # analysis saved here # preference flows parties = ['L/NP', 'ALP', 'GRN', 'OTH'] preference_flows_2010 = [0.9975, 0.0, 0.2116, 0.5826] preference_flows_2013 = [0.9975, 0.0, 0.1697, 0.5330] preference_flows_2016 = [0.9975, 0.0, 0.1806, 0.5075] n_parties = len(parties) # polling data workbook = pd.ExcelFile('./Data/poll-data.xlsx') df = workbook.parse('Data') # drop pre-2016 election data df['MidDate'] = [pd.Period(d, freq='D') for d in df['MidDate']] df = df[df['MidDate'] > pd.Period('2016-07-04', freq='D')] # push One Nation into Other df['ONP'] = df['ONP'].fillna(0) df['OTH'] = df['OTH'] + df['ONP'] # set start date start = df['MidDate'].min() - 1 # the first date is day 1 df['Day'] = df['MidDate'] - start # day number for each poll n_days = df['Day'].max() # maximum days n_polls = len(df) # set discontinuity date - Turnbull's last day in office discontinuity = pd.Period('2018-08-23', freq='D') - start # UPDATE stability = pd.Period('2018-10-01', freq='D') - start # UPDATE # manipulate polling data ... y = df[parties] center = 100 centreing_factors = center - y.mean() y = y + centreing_factors # add polling house data to the mix # make sure the "sum to zero" exclusions are # last in the list houses = df['Firm'].unique().tolist() exclusions = ['YouGov', 'Ipsos'] # Note: we are excluding YouGov and Ipsos # from the sum to zero constraint because # they have unusual poll results compared # with other pollsters for e in exclusions: assert(e in houses) houses.remove(e) houses = houses + exclusions map = dict(zip(houses, range(1, len(houses)+1))) df['House'] = df['Firm'].map(map) n_houses = len(df['House'].unique()) n_exclude = len(exclusions) # sample metrics sampleSize = 1000 # treat all polls as being of this size pseudoSampleSigma = np.sqrt((50 * 50) / sampleSize) # --- compile model # get the STAN model with open ("./Models/primary simultaneous model.stan", "r") as f: model = f.read() f.close() # encode the STAN model in C++ sm = stan_cache(model_code=model) # --- fit the model to the data ct_init = np.full([n_days, n_parties], center*1.0) def initfun(): return dict(centre_track=ct_init) chains = 5 iterations = 2000 data = { 'n_days': n_days, 'n_polls': n_polls, 'n_houses': n_houses, 'n_parties': n_parties, 'pseudoSampleSigma': pseudoSampleSigma, 'centreing_factors': centreing_factors, 'centered_obs_y': y.T, 'poll_day': df['Day'].values.tolist(), 'house': df['House'].values.tolist(), 'n_exclude': n_exclude, 'center': center, 'discontinuity': discontinuity, 'stability': stability, # let's set the day-to-day smoothing 'sigma': 0.15, 'sigma_volatile': 0.4, # preference flows at past elections 'preference_flows_2010': preference_flows_2010, 'preference_flows_2013': preference_flows_2013, 'preference_flows_2016': preference_flows_2016 } fit = sm.sampling(data=data, iter=iterations, chains=chains, init=initfun, control={'max_treedepth':13}) results = fit.extract() # --- check diagnostics print('Stan Finished ...') import pystan.diagnostics as psd print(psd.check_hmc_diagnostics(fit)) # --- save the analysis with open(intermediate_data_dir + 'cat-' + 'output-primary-zero-sum.pkl', 'wb') as f: pickle.dump([df,sm,fit,results,data, centreing_factors, exclusions], f) f.close()

## Saturday, November 10, 2018

### Updated poll aggregates

I have not done this for a while ... so here goes.

First the two-party preferred (TPP) aggregation. While there was substantial movement associated with the leadership change (late August), and some volatility thereafter during September, things have settled down to around or just below where they were for the Coalition at its worst under the former Prime Minister.

Note: for the TPP aggregation, all pollsters are included in the constraint of setting house effects such that they sum to zero.

A quick comparison with my house-effect adjusted moving average models (which don't have discontinuities built in).

Since upgrading to Stan 2.18, my former primary vote model stopped working. So I have simplified and refactored this model. As a result of the refactoring, the requirement that the four parties total to 100 per cent on every day is not longer enforced in the model. Notwithstanding this, the total is typically within in 0.2 percentage points of 100 per cent (a mean of 99.99 per cent and a standard deviation of 0.11 percentage points).

I have also taken YouGov and Ipsos out of the sum-to-zero constraint for house effects (because they were producing primary vote estimates that differed substantially from other pollsters). As a consequence, these excluded pollsters only inform the shape of the aggregation. Their polls do not inform the vertical positioning of the aggregate on the charts.

Of note:

The medians from the above plots can be compared. The following plots have the medians adjusted so that they sum to 100 per cent exactly on each and every day.

And finally, a quick look at One Nation (which is included in "other" above). Note: I have excluded Ipsos from this analysis as its treatment of One Nation is inconsistent over time when compared with the other pollsters.

First the two-party preferred (TPP) aggregation. While there was substantial movement associated with the leadership change (late August), and some volatility thereafter during September, things have settled down to around or just below where they were for the Coalition at its worst under the former Prime Minister.

Note: for the TPP aggregation, all pollsters are included in the constraint of setting house effects such that they sum to zero.

A quick comparison with my house-effect adjusted moving average models (which don't have discontinuities built in).

Since upgrading to Stan 2.18, my former primary vote model stopped working. So I have simplified and refactored this model. As a result of the refactoring, the requirement that the four parties total to 100 per cent on every day is not longer enforced in the model. Notwithstanding this, the total is typically within in 0.2 percentage points of 100 per cent (a mean of 99.99 per cent and a standard deviation of 0.11 percentage points).

I have also taken YouGov and Ipsos out of the sum-to-zero constraint for house effects (because they were producing primary vote estimates that differed substantially from other pollsters). As a consequence, these excluded pollsters only inform the shape of the aggregation. Their polls do not inform the vertical positioning of the aggregate on the charts.

Of note:

- The leadership turmoil in late August and into September harmed the Coalition's primary vote, and other parties primary vote and helped the primary vote share for Labor and the Greens.
- After the volatility of the leadership change, that is to say since the start of October, Labor's primary vote has strengthened; while the Coalition's primary vote has been stagnant.

The medians from the above plots can be compared. The following plots have the medians adjusted so that they sum to 100 per cent exactly on each and every day.

And finally, a quick look at One Nation (which is included in "other" above). Note: I have excluded Ipsos from this analysis as its treatment of One Nation is inconsistent over time when compared with the other pollsters.

## Sunday, October 21, 2018

### The betting market on the morning after Wentworth

Some movement on the betting market overnight ...

House | Coalition Odds ($) | Labor Odds ($) | Coalition Win Probability (%) | |
---|---|---|---|---|

2018-10-20 | BetEasy | 3.20 | 1.30 | 28.89 |

2018-10-21 | BetEasy | 4.00 | 1.20 | 23.08 |

2018-10-20 | Ladbrokes | 3.25 | 1.28 | 28.26 |

2018-10-21 | Ladbrokes | 3.50 | 1.25 | 26.32 |

2018-10-20 | Sportsbet | 3.20 | 1.30 | 28.89 |

2018-10-21 | Sportsbet | 3.50 | 1.25 | 26.32 |

2018-10-20 | William Hill | 3.40 | 1.30 | 27.66 |

2018-10-21 | William Hill | 3.40 | 1.30 | 27.66 |

## Monday, October 15, 2018

### Polling update

Today we had an Ipsos poll (45-55 in Labor's favour) and a Newspoll (47-53) with vastly different interpretations. For Newspoll the story was one of steady Coalition improvement (47 is better than Morrison's debut at 44). For Ipsos it was one of no benefit from the recent leadership change. The last Turnbull poll under Ipsos was also 45-55.

In the Bayesian model, I have allowed for a discontinuity in public opinion on 23 August, and for a period of higher than normal volatility in day-to-day voting sentiment from 24 August to 1 October. The results are as follows.

Enough time has passed since the Coalition leadership change for the moving average models to start to come back into alignment with the Bayesian model. Not withstanding the Coalition bounce following the immediate polling collapse in reaction to the leadership change in August 2018, Coalition voting sentiment is as low now as it was at Turnbull's worst period in the polls in late 2017.

My primary vote model has decided to stop working. Actually, I upgraded to the latest versions of Stan and pystan, and I need to tweak the model to get it working again.

In the Bayesian model, I have allowed for a discontinuity in public opinion on 23 August, and for a period of higher than normal volatility in day-to-day voting sentiment from 24 August to 1 October. The results are as follows.

Enough time has passed since the Coalition leadership change for the moving average models to start to come back into alignment with the Bayesian model. Not withstanding the Coalition bounce following the immediate polling collapse in reaction to the leadership change in August 2018, Coalition voting sentiment is as low now as it was at Turnbull's worst period in the polls in late 2017.

My primary vote model has decided to stop working. Actually, I upgraded to the latest versions of Stan and pystan, and I need to tweak the model to get it working again.

## Wednesday, September 26, 2018

### Updated aggregation

With Essential coming in at 47-53 in Labor's favour, it is time to update the aggregation. Both Essential and Newspoll are pegging a two percentage point recovery for the Coalition over the past four weeks. I have tweaked the TPP model to allow for a more rapid changes in public opinion over the period immediately following the recent disruption event.

Turning to the primary vote model ...

Turning to the primary vote model ...

Subscribe to:
Posts (Atom)