Introducing PB.C monitoring on poll sample distortion

Here’s a statistic that everybody interested in political opinion polls should take into account: If you make a completely random unsolicited phone call and manage to persuade the person answering to tell you how they voted in the General Election there’s a 44.7% chance that it will be Labour.

This compares, of course, with the 36.2% of voters who did actually vote for the party last May – so the proportion telling pollsters they voted this way is 23.5% more than actually did so. This could completely skew polling results unless corrective action is taken.

Your random call will produce a 28.3% chance of being told that they voted Tory against 33.2% that actually did so and a Lib Dem share of 20.3% against the 22.7% actual.

These figures are based on comprehensive, and what will be ongoing, PB.C monitoring that aggregates the raw past voting answer from every single phone poll since May 2005 where there is publicly available data.

Of course not everyone who is surveyed can remember what they did last May and this is factored in by the polling organisations that do weight by past vote recall. The pollsters also weight by socio-economic and demographic factors before coming up with their headline numbers.

The objective of the monitoring is to provide a yard-stick against which we can judge polling methodology.

All ICM and Populus surveys where a General Election voting intention question is asked adjust their figures in accordance with formulas based on how respondees answer the past vote recall question. ICM have pioneered this approach and deserve to be commended.

But be warned: sometimes these two pollsters carry surveys on political topics without asking voting intention and recall questions and my personal rule is to treat these findings with some caution.

Mori, which sometimes interview people face to face and sometimes uses the phone does not correct its finding in this way. What the firm does do is only to include those “certain to vote” in its headline figures.

Quite why Labour voters are more likely to answer the phone and agree to be interviewed I don’t know. But there’s now overwhelming evidence that they do and those who try to predict political outcomes should take this into account.

YouGov, which carries out surveys online amongst members of its polling panels, has its own “party identifier” weighting system designed to prevent distorted samples.

Mike Smithson