Are we now in an era of more accurate polling?

At the 2005 General Election I made several thousand pounds on a number of spread bets “selling” the projected Labour share. The spreads were very much in line with what the pollsters were reporting and I believed that they were over-stating the party. Thus one bet was at 38.5% against the actual 36.3% – meaning my winnings were the difference between the two numbers multiplied by the four figure sum I had bet per unit.

For Labour supporters present two great problems for pollsters. For whatever reason they are much more likely than supporters of other parties to answer the phone when the polling firm calls; and they are more likely to say they will turnout vote when they don’t.

The result if measures are not taken is that you get inflated Labour shares and distorted poll findings. The 1992 General Election was one of the worst for this when the final polls had Labour and the Tories neck and neck but John Major romped (if that’s the right word) home with an 8% margin.

Even in 1997, 2001 and to a much lesser extent 2005 there was some Labour over-statement and last year one pollster, NOP, got the actual vote shares spot on. These are some of the measures that are used:-

Past vote weighting. You ask how interviewees voted last time and compare the numbers with the actual result. ICM and Populus generally find that about 45% say they supported Labour against the actual of 36.2%. So in each poll they adjust the numbers they have downwards but not quite in line with what actually happened to deal with the misremembering factor. The ICM past vote weighting formula currently has a 6% Labour margin while the Populus one is closer to 9%. It was actually 3%.

Adjusting for turnout. Currently all the regular pollsters apart from YouGov adjust their final figures to take into account the way interviewees rate the likelihood of voting on a scale of, usually, 0-10. Mori, which does not use a past vote weighting calculation, only includes those “certain to vote” in its final figures. A key element is that the turnout adjustment is only used for the voting intention question – not the other elements in a poll such as rating the different leaders. One of the reason that I attach much less importance to these figures is that they include the opinions of a lot of non-voters.

Adjusting for differential turnout. In an interesting development for its 2005 final poll Mori made an adjustment to correct for differential exaggeration of propensity to vote (using a turnout adjustment which reduced by a third the extent to which the turnout gap between supporters of different parties had been narrowed since the start of the campaign. I assume that this cut their projected Labour share. I have not seen that on a Mori poll since.

Weighting by party identifier. YouGov’s political polls are restricted to members of its polling panel on whom the firm has a mass of data. This includes responses to question on how people voted at the General Election that were asked within a few days of May 5th 2005. The firm also records the party that people “identify” with and correlates those to how they actually voted.

We will know that the pollsters have finally cracked the problems when one of them understates the Labour share when tested against real results.

One consquence of all these measures is that we are now seeing much less variation in the different surveys and, I believe, more confidence in what is being reported.

Mike Smithson