|  |
|
|
“The polls are all over the place!” tweeted Jeremy Vine excitedly. It was the tail-end of last week and we’d just published our latest voting intention poll. With a one point Tory lead it presented a very different picture from the three polls already splashed on the next day’s Guardian showing Labour well ahead.
Mr Vine’s sentiment was revealing only half the story though: yes different polls were showing different results, but “all over the place” they were not. The polls have instead been telling a consistent story for some time, but not the story that many think.
While there has been some astonishment at recent variation in the polls, many poll watchers have been cautious, citing natural sample variation and cautioning “look at the trend”. It is the same sentiment which has led to a number of “poll of polls” springing up in the approach to the 2015 election.
These have had the race extremely close, with Labour a point or two ahead. As can be seen below with the BBC’s “poll tracker”, Labour has been mostly in the lead for the past month, the Conservatives occasionally drawing level.
|
 |
The problem with these “polls of polls” though is that the average can get flooded with lots of information taken from only a handful of sources. By using all recently released polls, it sees mostly online surveys included in the pot as they are produced more frequently. Even models which try and take this into account by making sure each pollster only has one figure represented in the average end up leaning towards online surveys because most pollsters poll online and relatively few conduct regular telephone polling.
If we split out the two, the story becomes quite different.
Looking at the average of all online polls released by major pollsters this year, the picture looks similar to the various “polls of polls”: it is a tight race, with Labour nosing just in front. The party is on a shade over 33.6% while the Conservatives are just above 32.6% (with an average we can permit ourselves the excess of a decimal point).
Looking at the telephone polling only, the race is still tight, but crucially the two parties are in the reverse order: the Conservatives leading Labour 33.4% to 32.2%.
Some may say it all just means everything is close, which it no doubt is. But it is worth bearing in mind it is not a “tie”. The 1.4% point difference between the Labour figure in telephone polls (32.2%) and online polls (33.6%) is larger than the margin of error that completely random samples would imply (0.62 on 25,000 telephone interviews, 0.2 on 250,000+ online interviews).
Looking at individual results reveals the same systematic difference. Of the 131 online polls conducted this year, they split towards Labour leads by more than two to one (78 to 28). The 25 telephone polls show the reverse: more than twice as many show a Conservative lead as show a Labour one (14 to 6). The picture couldn’t be clearer: online polls show Labour ahead, telephone polls the Tories ahead.
These findings also have an impact on the much talked about “crossover” – the point at which the Conservatives will overtake Labour. This has apparently been put off and off by Tory HQ “like an apocalyptic tribe that mistakenly forecasts the end of the world”. But by focusing on a “poll of polls” skewed towards online surveys, we might already have missed the actual “crossover” point.
Labour have failed to lead in a ComRes telephone poll yet this year. In the monthly average of all phone polls, this is also the case: the Conservatives led on average by 0.6% in January, 0.7% in February and 1.5% in March. According to phone polls, “crossover” was reached at the beginning of the year and the Conservatives have led ever since.
|
 |
This is not to say that the race is not still a hot contest: Labour may still fight their way into a lead (even if historical precedent would suggest it’s unlikely). Similarly, given the vagaries of turning votes into seats, Labour may still end up with most MPs, even if they do not have the most ballots cast for them. But for the time being, telephone polls have the Tories ahead on votes.
Online vs. telephone is not a new question, indeed, Nate Silver’s FiveThirtyEight websitehas tried answering it, so too has The Guardian.
We’ve been conducting voting intention both online and by telephone for the past five years, and up to now they’ve told a similar story throughout. In the heat of an election campaign though, people signed up to online panels may be at risk of being over-surveyed: bombarded with voting intention questions and information about the campaign in order to get their opinion of it. In this instance, for the time being at least, telephone polling of randomised landline and mobile numbers becomes your photographer’s dark room: used when you need to get the very crispest definition between your colours but requiring lots of time and attention. By polling day, telephone polls were most accurate in the 2010 General Election, the AV referendum in 2011 and at the Scottish referendum last year.
Does this necessarily mean that one method is more valid than another? No. All the major pollsters are committed to accuracy and transparency – it’s the lifeblood of their businesses. But it may be worth taking bearing in mind how a poll was conducted before deciding what it means for the overall trend.
After the event, the outcome of elections often appears to be clear and long in the making. If the Conservatives end up on top this time around, it may not be because of “late swing” or “shy Tories”, it may have been staring us in the face for weeks or months already.
|
|
No comments:
Post a Comment
Comment is open to all feel free to link to this blog.