Canadian pollsters advise voters to be wary of polls!

Research-live, the online newspaper for the UK market research industry reports that the Marketing Research & Intelligence Association (MRIA) in Canada has been driven to take out a full-page advertisement in the The Hill Times (a political weekly) aimed at reversing criticism of opinions polls.

It seems the ad is needed because the pollsters themselves have been blabbing about the state of their own industry. Some it seems have been quoted lamenting the proliferation of polls, a decline in standards (pollsters and journalists) and excessive competition.

The criticism in Canada is that pollsters provide an almost daily fix (of vote intention numbers) for media junkies, who “breathlessly report minor fluctuations” that are statistically insignificant. Pollsters themselves are concerned that they are feeding a media addiction to “political horse race numbers”.

The worry for Canadian pollsters is not simply that there are just too many polls; they are also said to suffer from a “combination of methodological problems, commercial pressures and an unhealthy relationship with the media”.

Random samples require all voters to have an equal chance of being selected and on that basis margins of error can be applied The criticism in Canada is that the methodological foundations of political polls have been swept away by declining response rates for telephone polls (15% in Canada) and increased use of volunteer online panels. Telephone pollsters weight to compensate for the fact that they have difficulty reaching some groups which Canadian critics say amounts to “messing around with random probability theory”.

Meanwhile Canadian pollsters also point out the obvious problem that people who volunteer to join a pollster’s online panel and are paid to participate are unlikely to amount to a representative sample of the whole population. Interestingly Canada’s MRIA bans the reporting of margins of error for online polls altogether as it contends they are misleading!

In Canada, in the 1970’s there were a handful of polling companies; polls were expensive so media outlets bought carefully. Pollsters complain that “now almost anyone can profess to be a pollster, with little or no training. There is so much competition that polls are given away free, in hopes the attendant publicity will boost business” (Alan Gregg, chairman of Harris-Decima). As a result, it is claimed that pollsters don’t care much for the quality, often throwing a couple of questions into the middle of an omnibus survey. In Canada the complaint of some is that there is “a competitive auction to the bottom”.

Meanwhile, the media in Canada need to make exciting news out of poll figures, even when the story is not supported by the statistics. And it seems that the Canadian polling industry has little interest in taming the “hype” having “fallen in love with the sound of its own voice”.

Some fear that the sheer volume of polls in Canada is in danger of creating a situation in which the media and therefore voters “just want to know the score for the game; they don’t want to watch the game”.

No wonder the MRIA reason they need an advertising campaign to try and counter what looks like a devastating attack on the polling industry from within its own ranks.

Now for the really big question; the elephant in the room; the one that you would prefer I didn’t ask……….. How many of these criticisms could be levelled at the UK polling industry and journalists who report the findings?

There, I said it!

Nick Sparrow is the former head of polling at ICM – he writes a monthly column on PB