A guest slot from Nick Sparrow – former head of polling at ICM

In a search for more accurate vote intention estimates following the debacle of 1992, one modification we made was to prompt respondents with the names of the main political parties, Conservative, Labour, Liberal Democrat (and SNP or Plaid Cymru in relevant areas). The prompts reminded people of the existence of the Liberal Democrats – a partly forgotten alternative between elections. The prompts also may have had the effect of making it a bit easier for some people to admit being Conservative supporters as this alternative had been mentioned by the interviewer.

But we found that instructing telephone interviewers to prompt with the names of any minor party led to an over-estimate of their vote share, not mentioning them led to an under-estimate. With other parties accounting for rather less than 5% of the vote this was a small problem. A similar issue has since confronted online pollsters as all possible answers must be listed on screen. One option (among many) to avoid inflating vote shares for minor parties is to have all main alternatives on the first screen and route those who select an “other” party to a second screen and list.

But, of course, this is no longer a small problem, but – apparently – a great big one. In real elections the proportion of people voting for minor parties has been increasing steadily, and particularly so since the early 1990s. If any of the most recent polls really reflect voter intentions then we have to believe there has been an explosion in support for other parties, notably UKIP, since 2010.

The established conventions for question design in market research would be to ask for a spontaneous response, i.e without mentioning any possible choices, or prompt with all the main alternatives.  By this yardstick it becomes difficult to justify continuing to omit mention of UKIP at least, and arguably other smaller parties as well, as they appear to add up to a choice for almost one in five of those who would vote in a new election.

Unfortunately, this problem is compounded by another.  With the next election 2 years or so away, pollsters pose a question that is presently absurdly hypothetical.  Ask a respondent “If there were to be an immediate general election …..” and many will immediately be thinking “well there isn’t!”  So for some the question presents an opportunity to have a rant at the Government, the opposition and in fact all those who govern us – including and increasingly the EU.  Now, 2 years before the next election, the temptation for some would be to express dissatisfaction by declaring support for a party that most effectively conveys that message.  Mention UKIP or the BNP by name (or list them alongside the main parties) and pollsters risk encouraging the tendency, thereby helping to further inflate claimed support for an option that, come the real election, may ebb away.

Once upon a time substantial element of Liberal Democrat support came from those who saw them as the “neither of the above” party.  At present they are very much part of the above, and that constraint very much helps other parties like UKIP.  Only during the next election campaign will the LibDems be able to present themselves as a truly separate party with their own vision for Britain’s future.

As the next election approaches I suggest the tendency of some to want to use the vote intention question to send a message of dissatisfaction to those who govern us will slowly be replaced by a more measured, and in some cases reluctant, verdict on which party and party leader has the best and most credible plan for the next 5 years.  Principally they will ask themselves which party is going to make them and their families better off.  Only then will we be able to see whether support for UKIP and others has been real, or a mirage.

So when looking at any new poll, here is what I would do …

Always consider the question that has been asked and possible responses offered to respondents and only then try and work out what the answers might mean.  Ask a stupid question ……

Consider whether the answers given, may not always be to the question that has been asked.
 
Unbelievable poll ratings for any political party usually are.

Whatever the polls presently say, come the real election expect things to be rather different

And always remember, it’s the economy, stupid

Nick Sparrow