A tale of two by elections – and why polls are snapshots, not predictions

 This was originally posted on Lord Ashcrofts Blog

By Lord Ashcroft
On 28 August Douglas Carswell announced he was leaving the Conservative Party and joining UKIP. Five days later I published a poll of the Clacton constituency that put his support in the resulting by-election at 56%, with the Tories on 25%, Labour on 16%, and the Lib Dems and Others on 2% each: a 32-point Carswell lead.
The results on by-election day six weeks later were remarkably similar to those in my survey. Carswell held the seat by 35 points. He won 59.7% of the vote, with the Conservatives second on 24.6%, Labour on 11.2%, the Lib Dems on 1.4% and Others on 3.0%.
But on the same day, Labour won the Heywood & Middleton by-election by just two points – although a poll I conducted the previous weekend found the party with a 19-point lead over UKIP. Why the discrepancy?
One possible reason is straightforward sample error – the inevitability that every so often every pollster will come up with a rogue result. This seems unlikely given that four days earlier, Survation published a poll of the constituency which also put Labour 19 points ahead. It would be remarkable if two separate organisations encountered the same statistical gremlins in the same week, in the same place, producing the same finding.
Another possibility is that the methodology of my poll was flawed. This is something that pollsters should be ready to consider when the evidence merits it, but I doubt it is the explanation in this case. For one thing, the poll from Survation, whose methodology in constituency surveys has important differences from my own, after all found the same big Labour lead.
Moreover, if the methodology is wrong, why did my polls manage to get so close to the result in Clacton? Or in Newark, where my by-election poll found each party within the margin of error of the final result?
My conclusion, then is that in Heywood & Middleton the state of play simply changed between the time the two polls were conducted and by-election day itself.
The Clacton by-election was a major event from the very beginning; I suspect most voters knew as soon as they heard about it whether they were with Carswell or against him. Few changed their minds, despite the vigorous campaign.
In Heywood & Middleton there was no dramatic defection to galvanise opinion at the outset, and no compelling reason for voters to focus on the contest early. This left the door open to the late UKIP surge that has often characterised the campaigns in which the party started as outsiders. Indeed John Bickley, the UKIP candidate, says the movement was such that with another two or three days he would have won.
This may seem a rather convenient explanation for a pollster, but that does not make it any less likely to be true. Though does not always happen, opinion really can shift in the last few days of a campaign. That is why, as I have always said, polls are snapshots, not predictions.