A guest slot by Bob Worcester – the founder of MORI

(This is a specially adapted version for PB of an article produced by Bob Worcester for his new Vox Populi blogs on the Ipsos MORI site. The aim is to produce a monthly piece between now and the next election based on Bob’s personal experience of nearly 40 years conducting and observing polls and how they are reported)

“Let’s look at the record”, as the politicians say. Just this past month we’ve seen two polls early in August, ICM in the Sunday Express and BPIX in the Sunday Mail, both being conducted over the same few days, yet one had a 15 point lead for the Conservatives (ICM) and the other had a 24 point lead (BPIX). They can’t both be right I hear the poll pickers say.

Or can they?

Then over identical days in mid-month, 15th-17th, ICM for the Guardian had 15 point Tory lead again, while Ipsos MORI had 24, published by the Press Association. Well that proves it, doesn’t it?

Or does it?

No, it doesn’t, if you look at the share, not the lead. For polls are subject to the laws of statistics (as any statistician will tell you, if you ask them), and polls using properly designed samples, and we all do, operate within a so-called ‘confidence interval’ of plus or minus three percent for a sample of c. 1,000 people. And the four polls above, plus the three done by YouGov in between them, are all, seven out of seven, within a three point ‘margin of error’ of the average share for the Conservatives, Labour, Liberal Democrats, and Others.

That’s as good as it gets folks, and if you double the sample size, you don’t improve the accuracy much. And that holds whether you’re talking about a poll in America’s 240 million potential voters, Britain’s 45 million, or Trinidad’s 1.3 million or even the Maldives’ 200,000 electorate, holding their first ever contested presidential election this coming month. Hard to believe. Yeah, I know, but that’s the way it works.

Still don’t believe me?

Take all the polls in the first quarter of 2008. There were 23 polls published by one or another of the seven pollsters making the running. The Tories? 23 out of 23 were within the three percent margin as were the Liberal Democrats. Labour? 20 out of 23, as were the collective others. So in the first quarter of this year, 93% of the 92 party shares were within the plus or minus three percent margin. And what does statistics tell us? That the +/-3% rule you can expect to work 95% of the time if done at the same time, and these 23 polls covered three months!

OK, a fluke? What about April? 92%. May 84% (not too good, shape up pollsters). June? 94% (that’s better). July? 97% (wow). August so far? 100%! Seven polls out of seven hit the average for the month (to date), the Tories at 46%, Labour 26%, Liberal Democrats 17% and Others 11% within plus or minus three percent margins.

So why oh why do most folks believe the polls are all over the place. Some folks don’t believe in them at all, any more than Quiji boards or reading tea leaves. Another reason is that most folks aren’t statisticians. Fair enough, but most of those commenting on the polls in the newspapers, on radio and television, and certainly many who comment on them in the blogs and their responders, don’t really understand how they work, why those of us in the trade believe in them (otherwise we wouldn’t bother doing them, as nobody makes much money on them and they divert us from doing more profitable business).

And if you’re wondering, of the 65 polls published since the beginning of the year, the accuracy score is a credible 93.5% against a 95% out of 100% target for a perfect score: 98% – damn near perfect. Do the weather people do so well? Punters? Do you?

If you want the numbers, email me. I’d be happy to send the evidence winging back. The full table is on our web site, and we’ll be keeping a running total from here on right up to the next election.

Keep taking the polls…

Bob Worcester