Have the polls really got better?

Whenever we’ve criticised polling accuracy apologists for the industry have rushed to their defence saying that techniques have improved and things have got better since 2001 when the average overstatement of the Labour lead was 6.6%.

But in a recent example since then, at the Scottish Parliament Elections last year, the overstatement by the conventional pollsters of the LAB-CON margin was two percentage points bigger than the national polls at the 1992 General Election, which itself was one of the worst polling performances ever.

One final week survey by a major national firm put the Tories on 10% against the 16.6% that was actually achieved. The works out at a “loss” of two out of five. The other two interviewer-based pollsters recorded 12% – thus “missing” more than 1 Tory in 4.

The pollster to come out of Scotland 2003 with its reputation most intact was the internet survey by YouGov which over-estimated Labour by 1.4% and underestimated the Tories by 0.6%.

YouGov, of course, do not use interviewers which we believe are at the heart of many of the polling problems. People respond to people and at election after election it has been shown that Tories find it hard to admit their allegiance.

Maybe in Scotland last year it was even harder to tell another human being that you supported what was then IDS’s party – hence the huge proportion of missing Tories.

We are not naming the conventional polling firms here because we are making a general point about the use of interviewers. A problem with YouGov is that those that are polled are self-selected members of the firm’s “polling club” who get paid for taking part. For each survey the firm decides who shall take part and it does this on the basis of information supplied when people join. This, it is argued, makes their samples less random.

All this reinforces our scepticism about the current poll ratings. We are certainly not suggesting that current national polls are like the Scottish ones and losing one in four Tories but we think that Michael Howard’s party is doing much better than current figures suggest.

We think that the future of polling will be based on automation where you take away the personal dimension of the interviewer. This replicates more closely the conditions of a secret ballot and the UK experience of such an approach has been very positive.

Mike Smithson