Can the pollster be accused of “persistently understating” his ratings?

With the row over Ken Livingstone’s custodianship of the London mayoralty continuing unabated the election on May 1st that will determine his future looks set to be the biggest UK political betting bonanza since the 2005 general election.

It has all the ingredients that make it attractive to punters – high profile contenders who everybody knows by their first names; passionate feelings across the political spectrum about the current incumbent and the prospect of a really tight race.

The big question if you are having a bet is how much can you trust can you trust the polls – particularly the ones from YouGov which looks set to carry out most of the surveys. When their first London poll of 2008 showed that only one point separated Ken and Boris there was an attempt in some quarters to rubbish the findings.

A spokesperson for the Mayor said: “Online polling has persistently radically understated Ken Livingstone’s support. A YouGov poll on the eve of the last mayoral election gave Ken Livingstone a lead of 1% whereas in the event Ken won by 11%.”

If that was the case then it would undermine punter confidence in the figures. So let’s go back to what actually happened four years ago.

On the day before the 2004 vote the Evening Standard carried a report of its survey from the pollster which found that amongst all potential electors Livingstone was beating Norris by 37% to 26%. When the second preference votes were taken into account, YouGov found a split of 55% to 45%.

The actual result after the second preferences were counted was Livingstone 55.4% to Norris’s 44.6% – which meant that YouGov was almost exactly dead on.

Where the confusion has come is that the paper also carried a figure of those saying they were “certain to vote” which had a Ken margin of 2%. These were the numbers that were splashed and the ones that got remembered. But this tight margin excluded respondents who had said they were “very likely to vote” or “would probably vote” which, I would suggest, distorts the overall figures.

YouGov is the only mainstream pollster that does not normally filter by certainty to vote.

In a poll on the election three weeks earlier YouGov found that Livingstone was winning by 35% to 23% – again a gap quite close to the winning margin. Criticism about last time should not be focussed at YouGov but at the Evening Standard’s coverage. That’s why I always say look at the polling details and not the news reports.

My concern over the 2008 poll is that the fieldwork took place over the holiday period which can have a distorting effect. Mark Senior, who posts here regularly on polling matters, has pointed out that there was a massive over-representation amongst the responders in the 55+ age groups and this had to be scaled back drastically.

What we need is a new poll. Latest Mayoral betting is here.

Mike Smithson