This page provides some technical background on the Bayesian poll aggregation models used on this site. The page has been updated for the 2016 Federal election.
General overview
The aggregation or data fusion models I use are known as hidden Markov models. They are are sometimes referred to as state space models or latent process models.I model the national voting intention (which cannot be observed directly; it is "hidden") for each and every day of the period under analysis. The only time the national voting intention is not hidden, is at an election. In some models (known as anchored models), we use the election result to anchor the daily model we use.
In the language of modelling, our estimates of the national voting intention for each day being modeled are known as states. These "states" link together to form a Markov process, where each state is directly dependent on the previous state and a probability distribution linking the states. In plain English, the models assume that the national voting intention today is much like it was yesterday. The simplest models assume the voting intention today is normally distributed around the voting intention yesterday.
The model is informed by irregular and noisy data from the selected polling houses. The challenge for the model is to ignore the noise and find the underlying signal. In effect, the model is solved by finding the the day-to-day pathway with the maximum likelihood given the known poll results.
To improve the robustness of the model, we make provision for the long-run tendency of each polling house to systematically favour either the Coalition or Labor. We call this small tendency to favour one side or the other a "house effect". The model assumes that the results from each pollster diverge (on average) from the from real population voting intention by a small, constant number of percentage points. We use the calculated house effect to adjust the raw polling data from each polling house.
In estimating the house effects, we can take one of a number of approaches. We could:
- anchor the model to an election result on a particular day, and use that anchoring to establish the house effects.
- anchor the model to a particular polling house or houses; or
- assume that collectively the polling houses are unbiased, and that collectively their house effects sum to zero.
Typically, I use the first and third approaches in my models. Some models assume that the house effects sum to zero. Other models assume that the house effects can be determined absolutely by anchoring the hidden model for a particular day or week to a known election outcome.
There are issues with both approaches. The problem with anchoring the model to an election outcome (or to a particular polling house), is that pollsters are constantly reviewing and, from time to time, changing their polling practice. Over time these changes affect the reliability of the model. On the other hand, the sum-to-zero assumption is rarely correct. Of the two approaches, anchoring tends to perform better.
Another adjustment we make is to allow discontinuities in the hidden process when either party changes its leadership. Leadership changes can see immediate changes in the voting preferences. For the anchored models, we allow for discontinuities with the Gillard to Rudd and Abbott to Turnbull leadership changes. The unanchored models have a single discontinuity with the Abbott to Turnbull leadership change.
Solving a model necessitates integration over a series of complex multidimensional probability distributions. The definite integral is typically impossible to solve algebraically. But it can be solved using a numerical method based on Markov chains and random numbers known as Markov Chain Monte Carlo (MCMC) integration. I use a free software product called JAGS to solve these models.
The dynamic linear model of TPP voting intention with house effects summed to zero
This is the simplest model. It has three parts:- The observational part of the model assumes two factors
explain the difference between published poll results (what we observe)
and the national voting intention on a particular day (which, with the
exception of elections, is hidden):
- The first factor is the margin of error from classical statistics. This is the random error associated with selecting a sample; and
- The second factor is the systemic biases (house effects) that affect each pollster's published estimate of the population voting intention.
- The temporal part of the model assumes that the actual population voting intention on any day is much the same as it was on the previous day (with the exception of discontinuities). The model estimates the (hidden) population voting intention for every day under analysis.
- The house effects part of the model assumes that house effects are distributed around zero and sum to zero.
This model builds on original work by Professor Simon Jackman. It is encoded in JAGS as follows:
model {
## -- draws on models developed by Simon Jackman
## -- observational model
for(poll in 1:n_polls) {
yhat[poll] <- houseEffect[house[poll]] + hidden_voting_intention[day[poll]]
y[poll] ~ dnorm(yhat[poll], samplePrecision[poll]) # distribution
}
## -- temporal model - with one discontinuity
hidden_voting_intention[1] ~ dunif(0.3, 0.7) # contextually uninformative
hidden_voting_intention[discontinuity] ~ dunif(0.3, 0.7) # ditto
for(i in 2:(discontinuity-1)) {
hidden_voting_intention[i] ~ dnorm(hidden_voting_intention[i-1], walkPrecision)
}
for (j in (discontinuity+1):n_span) {
hidden_voting_intention[j] ~ dnorm(hidden_voting_intention[j-1], walkPrecision)
}
sigmaWalk ~ dunif(0, 0.01)
walkPrecision <- pow(sigmaWalk, -2)
## -- house effects model
for(i in 2:n_houses) {
houseEffect[i] ~ dunif(-0.15, 0.15) # contextually uninformative
}
houseEffect[1] <- -sum( houseEffect[2:n_houses] )
}
Professor Jackman's original JAGS code can be found in the file kalman.bug, in the zip file link on this page, under the heading Pooling the Polls Over an Election Campaign.
The anchored dynamic linear model of TPP voting intention
This model is much the same as the previous model. However, it is run with data from prior to the 2013 election to anchor poll performance. It includes two discontinuities, with the ascensions of Rudd and Turnbull. And, because it is anchored, the house effects are not constrained to sum to zero.model {
## -- observational model
for(poll in 1:n_polls) {
yhat[poll] <- houseEffect[house[poll]] + hidden_voting_intention[day[poll]]
y[poll] ~ dnorm(yhat[poll], samplePrecision[poll]) # distribution
}
## -- temporal model - with two or more discontinuities
# priors ...
hidden_voting_intention[1] ~ dunif(0.3, 0.7) # fairly uninformative
for(j in 1:n_discontinuities) {
hidden_voting_intention[discontinuities[j]] ~ dunif(0.3, 0.7) # fairly uninformative
}
sigmaWalk ~ dunif(0, 0.01)
walkPrecision <- pow(sigmaWalk, -2)
# Up until the first discontinuity ...
for(k in 2:(discontinuities[1]-1)) {
hidden_voting_intention[k] ~ dnorm(hidden_voting_intention[k-1], walkPrecision)
}
# Between the discontinuities ... assumes 2 or more discontinuities ...
for( disc in 1:(n_discontinuities-1) ) {
for(k in (discontinuities[disc]+1):(discontinuities[(disc+1)]-1)) {
hidden_voting_intention[k] ~ dnorm(hidden_voting_intention[k-1], walkPrecision)
}
}
# after the last discontinuity
for(k in (discontinuities[n_discontinuities]+1):n_span) {
hidden_voting_intention[k] ~ dnorm(hidden_voting_intention[k-1], walkPrecision)
}
## -- house effects model
for(i in 1:n_houses) {
houseEffect[i] ~ dnorm(0, pow(0.05, -2))
}
}
The latent Dirichlet process for primary voting intention
This model is more complex. It takes advantage of the Dirichlet (pronounced dirik-lay) distribution, which always sums to 1, just as the primary votes for all parties would sum to 100 per cent of voters at an election. A weakness is a transmission mechanism from day-to-day that uses a "tightness of fit" parameter, which has been arbitrarily selected.The model is set up a little differently to the previous models. Rather than pass the vote share as a number between 0 and 1; we pass the size of the sample that indicated a preference for each party. For example, if the poll is of 1000 voters, with 40 per cent for the Coalition, 40 per cent for Labor, 11 per cent for the Greens and and 9 per cent for Other parties, the multinomial we would pass across for this poll is [400, 400, 110, 90].
More broadly, this model is conceptually very similar to the sum-to-zero TPP model: with an observational component, a dynamic walk of primary voting proportions (modeled as a hierarchical Dirichlet process), a discontinuity for Turnbull's ascension, and a set of house effects that sum to zero across polling houses and across the parties.
data {
zero <- 0.0
}
model {
#### -- observational model
for(poll in 1:NUMPOLLS) { # for each poll result - rows
adjusted_poll[poll, 1:PARTIES] <- walk[pollDay[poll], 1:PARTIES] +
houseEffect[house[poll], 1:PARTIES]
primaryVotes[poll, 1:PARTIES] ~ dmulti(adjusted_poll[poll, 1:PARTIES], n[poll])
}
#### -- temporal model with one discontinuity
tightness <- 50000 # kludge - today very much like yesterday
# - before discontinuity
for(day in 2:(discontinuity-1)) {
# Note: use math not a distribution to generate the multinomial ...
multinomial[day, 1:PARTIES] <- walk[day-1, 1:PARTIES] * tightness
walk[day, 1:PARTIES] ~ ddirch(multinomial[day, 1:PARTIES])
}
# - after discontinuity
for(day in discontinuity+1:PERIOD) {
# Note: use math not a distribution to generate the multinomial ...
multinomial[day, 1:PARTIES] <- walk[day-1, 1:PARTIES] * tightness
walk[day, 1:PARTIES] ~ ddirch(multinomial[day, 1:PARTIES])
}
## -- weakly informative priors for first and discontinuity days
for (party in 1:2) { # for each major party
alpha[party] ~ dunif(250, 600) # majors between 25% and 60%
beta[party] ~ dunif(250, 600) # majors between 25% and 60%
}
for (party in 3:PARTIES) { # for each minor party
alpha[party] ~ dunif(10, 250) # minors between 1% and 25%
beta[party] ~ dunif(10, 250) # minors between 1% and 25%
}
walk[1, 1:PARTIES] ~ ddirch(alpha[])
walk[discontinuity, 1:PARTIES] ~ ddirch(beta[])
## -- estimate a Coalition TPP from the primary votes
for(day in 1:PERIOD) {
CoalitionTPP[day] <- sum(walk[day, 1:PARTIES] *
preference_flows[1:PARTIES])
CoalitionTPP2010[day] <- sum(walk[day, 1:PARTIES] *
preference_flows_2010[1:PARTIES])
}
#### -- house effects model with two-way, sum-to-zero constraints
## -- vague priors ...
for (h in 2:HOUSECOUNT) {
for (p in 2:PARTIES) {
houseEffect[h, p] ~ dunif(-0.1, 0.1)
}
}
## -- sum to zero - but only in one direction for houseEffect[1, 1]
for (p in 2:PARTIES) {
houseEffect[1, p] <- 0 - sum( houseEffect[2:HOUSECOUNT, p] )
}
for(h in 1:HOUSECOUNT) {
# includes constraint for houseEffect[1, 1], but only in one direction
houseEffect[h, 1] <- 0 - sum( houseEffect[h, 2:PARTIES] )
}
## -- the other direction constraint on houseEffect[1, 1]
zero ~ dsum( houseEffect[1, 1], sum( houseEffect[2:HOUSECOUNT, 1] ) )
}
The anchored Dirichlet primary vote model
The anchored Dirichlet model follows. It draws on elements from the anchored TPP model and Dirichlet model above. It is the most complex of these models. It also takes the longest time to produce reliable results. For this model, I run 460,000 samples, taking every 23rd sample for analysis. On my aging Apple iMac and JAGS 4.0.1 it takes around 100 minutes to run.model {
#### -- observational model
for(poll in 1:NUMPOLLS) { # for each poll result - rows
adjusted_poll[poll, 1:PARTIES] <- walk[pollDay[poll], 1:PARTIES] +
houseEffect[house[poll], 1:PARTIES]
primaryVotes[poll, 1:PARTIES] ~ dmulti(adjusted_poll[poll, 1:PARTIES], n[poll])
}
#### -- temporal model with multiple discontinuities
# - tightness of fit parameters
tightness <- 50000 # kludge - today very much like yesterday
# Up until the first discontinuity ...
for(day in 2:(discontinuities[1]-1)) {
multinomial[day, 1:PARTIES] <- walk[day-1, 1:PARTIES] * tightness
walk[day, 1:PARTIES] ~ ddirch(multinomial[day, 1:PARTIES])
}
# Between the discontinuities ... assumes 2 or more discontinuities ...
for( disc in 1:(n_discontinuities-1) ) {
for(day in (discontinuities[disc]+1):(discontinuities[(disc+1)]-1)) {
multinomial[day, 1:PARTIES] <- walk[day-1, 1:PARTIES] * tightness
walk[day, 1:PARTIES] ~ ddirch(multinomial[day, 1:PARTIES])
}
}
# After the last discontinuity
for(day in (discontinuities[n_discontinuities]+1):PERIOD) {
multinomial[day, 1:PARTIES] <- walk[day-1, 1:PARTIES] * tightness
walk[day, 1:PARTIES] ~ ddirch(multinomial[day, 1:PARTIES])
}
# weakly informative priors for first day and discontinutity days ...
for (party in 1:2) { # for each minor party
alpha[party] ~ dunif(250, 600) # minors between 25% and 60%
}
for (party in 3:PARTIES) { # for each minor party
alpha[party] ~ dunif(10, 200) # minors between 1% and 20%
}
walk[1, 1:PARTIES] ~ ddirch(alpha[])
for(j in 1:n_discontinuities) {
for (party in 1:2) { # for each minor party
beta[j, party] ~ dunif(250, 600) # minors between 25% and 60%
}
for (party in 3:PARTIES) { # for each minor party
beta[j, party] ~ dunif(10, 200) # minors between 1% and 20%
}
walk[discontinuities[j], 1:PARTIES] ~ ddirch(beta[j, 1:PARTIES])
}
## -- estimate a Coalition TPP from the primary votes
for(day in 1:PERIOD) {
CoalitionTPP[day] <- sum(walk[day, 1:PARTIES] *
preference_flows[1:PARTIES])
}
#### -- sum-to-zero constraints on house effects
for(h in 1:HOUSECOUNT) {
for (p in 2:PARTIES) {
houseEffect[h, p] ~ dnorm(0, pow(0.05, -2))
}
}
# need to lock in ... but only in one dimension
for(h in 1:HOUSECOUNT) { # for each house ...
houseEffect[h, 1] <- -sum( houseEffect[h, 2:PARTIES] )
}
}
Beta model of primary vote share for Palmer United
A simplification of the Dirichlet distribution is the Beta distribution. I use the Beta distribution (in a similar model to the Dirichlet model above), to track the primary vote share of the Palmer United Party. This model does not provide for a discontinuity in voting associated with the change of Prime Minister.model {
#### -- observational model
for(poll in 1:NUMPOLLS) { # for each poll result - rows
adjusted_poll[poll] <- walk[pollDay[poll]] + houseEffect[house[poll]]
palmerVotes[poll] ~ dbin(adjusted_poll[poll], n[poll])
}
#### -- temporal model (a daily walk where today is much like yesterday)
tightness <- 50000 # KLUDGE - tightness of fit parameter selected by hand
for(day in 2:PERIOD) { # rows
binomial[day] <- walk[day-1] * tightness
walk[day] ~ dbeta(binomial[day], tightness - binomial[day])
}
## -- weakly informative priors for first day in the temporal model
alpha ~ dunif(1, 1500)
walk[1] ~ dbeta(alpha, 10000-alpha)
#### -- sum-to-zero constraints on house effects
for(h in 2:HOUSECOUNT) { # for each house ...
houseEffect[h] ~ dnorm(0, pow(0.1, -2))
}
houseEffect[1] <- -sum(houseEffect[2:HOUSECOUNT])
}
Code and data
I have made most of my data and code base available on Google Drive. Please note, this is my live code base, which I play with quite a bit. So, there will be times when it is broken or in some stage of being edited.What I have not made available is the Excel spreadsheets into which I initially place my data. These live in the (hidden) raw-data directory. However, the collated data for the Bayesian model lives in the intermediate directory, visible from the above link.
No comments:
Post a Comment