COMRES POLLWATCH - What do the polls actually tell us in Scotland?

POLLWATCH: What do the polls actually tell us in Scotland? With polls now causing the value of Sterling to tumble and panic mode setting in across Westminster, now is surely the time for a cool assessment of what they are actually saying. Are we witnessing an illusory Yes-mania or is this something very real? Let us start with a few things which definitely will not help you judge the state of play:       1. TV reports The media love a tight battle. From now until the date of the referendum this will be a “knife edge” contest with “everything still to play for”. Neutrality requirements mean that any TV report will include a mix of Yes and No supporters, and vox pop interviewees will play up to stereotypes.       2.  Social media Social media networks dramatically over-represent young people. They are a haven for activists, and a fertile environment for campaigns to generate the appearance of support. It is straightforward to express approval for something on social media – all you have to do is click “Like”. This does not always translate into votes.       3.  The word on the street Political canvassers often think their side is doing better than the polls suggest. Unfortunately for them, most people tailor their responses to their audience. Diehard activists are an extreme case, but how often have you nodded along to a taxi driver or a hairdresser whose views you completely oppose?       4.  Evidence from previous elections Elections and referendums are a bit like the weather. We can make general rules about seasons and climates that apply most of the time, but on a day-to-day basis we run into difficulties. The more specific the prediction, and the rarer the event, the harder it is to get it right. Predicting turnout One rule of thumb that has held steady across virtually all elections in the UK is the demographic profile of overall turnout: those who have voted before are significantly more likely to vote again; turnout tends to be higher among more affluent (ABC1) social classes; and the highest levels of turnout are among the over 65s. Even a record-breaking level of turnout in Scotland across all demographics is still likely to follow that same shape. For example, if final turnout were 80%, we would expect around 65-70% turnout among 16-24 year olds and around 90% turnout among over 65s. So what about the polls? Polling at least brings some science to the process. Pollsters ensure that their questions are carefully worded, that they are asked dispassionately, and that a large and statistically representative spread of people is questioned. Empirical evidence from censuses and previous elections is used to apply corrections to the data. But polling can still be susceptible to some of the same problems outlined above. Achieving the right balance of responses Like a TV journalist, pollsters need to obtain a balanced sample of viewpoints, but judging what that balance looks like can be tricky. Most try to avoid skewing towards a particular political outlook by weighting to “past vote” – in this case the 2011 Scottish Parliament election; but turnout at that election was only 50%, and it was an unusually good year for the SNP. Moreover, voters cast two votes in Scottish Parliament elections – one for a constituency representative and one for a party – making it doubly difficult to accurately recall how they voted. These factors have the potential to skew the party political balance of a sample. Exaggerating certainty The online pollsters can be affected by the same propensity to generate strong opinions as a social media feed, but this is usually factored in to any proper pollster’s methodology. This does not inherently bias the figures towards “Yes” or “No”, but it can make panellists more likely to overestimate certainty to vote. This may mean that lower turnout groups (like younger people and less affluent C2DEs) are having a disproportionate influence on some pollsters' results. 'Shy' respondents The face-to-face pollsters are afflicted by the same “interviewer effect” as the political canvasser. Despite asking their questions dispassionately, the presence of the interviewer (and sometimes the presence of other people in the background) can still make the question too awkward for the respondent to answer. This may explain why many more “don’t know” responses occur in the findings from face-to-face pollsters. There is some evidence that women are more likely than men to avoid answering a question in this way – not a problem if gender has little bearing on attitudes, but a distorting factor if men and women think differently about an issue. Ignore the “don’t knows” at your peril Most of the recent headlines have been generated by reports which have excluded “don’t know” respondents from the calculation. This gives a neat prediction of the final split. But it can be a misrepresentation of the polls. An analogy: if we asked 1000 people whether they prefer Coca-Cola or Pepsi, and 400 preferred Coca-Cola, 350 preferred Pepsi, and 150 had no strong opinion or were undecided, it would then be wilfully misleading to say that 53% of them prefer Coke. This is effectively the analysis that many journalists (and some pollsters) have been making. The truth is that while the two headline-grabbing, market-bashing polls of the week are each showing the same trend (an upward tick in “Yes” support), they are otherwise showing two very distinct pictures: The question is whether TNS-BMRB’s “don’t knows” will split evenly into “Yes” and “No”, or whether they are more likely to break towards one side. The evidence from previous referendums worldwide is that most “Don’t know” respondents will end up backing the “status quo” option on voting day. If that is the case, then “Yes” should be very concerned about their 39% figure this close to the vote. Some will argue that there no longer is a status quo option on the ballot – that we are now into Devo Max vs. Full Independence territory. But the former clearly involves less change and less of the unknown, and we should still expect cautious and undecided voters to lean this way on voting day. Keep your eye on the “don’t knows”.

Follow ComRes on Twitter for the latest polls and analysis:@ComResPolls Author: Andy White, Senior Consultant Be prepared for GE2015 with the new ComRes Election Toolkit: Battlebus The 2015 Battlebus is an online survey of 1,000 adults living in the 40 most marginal constituencies where Labour and the Conservatives share first and second place between them and battle head-to-head to get their candidate elected. This survey offers unique access to the opinions of those voters who will win or lose the election for the main parties - all at omnibus price levels. In the run up to the election, and whilst the parties are drafting their manifestos, this research tool is ideal for ensuring that each of the parties know the importance of your policy issues to those who will be decisive in getting them elected. This can be very powerful for lobbying material or for generating media hits. More information here. Future MPs survey Based on forecasts using our long-term voting intention surveys and careful psephological analysis, we have formulated a method of gauging the most likely composition of the House of Commons after May 2015. Guaranteeing a sample of 100 of those MPs and PPCs who are most likely to take a seat in Parliament after 2015, ComRes is offering its clients a chance to gain vital insight into the levels of support for policy issues post 2015, enabling organisations to be on the front foot for when the new Parliament sits. More information here