(I’m on holiday in Dorset at the moment and this guest slot submitted by Flockers echoes quite a lot of comments that we are getting whenever YouGov polls are published – which is now five times a week. When I return next week I’m hoping that the firm’s Peter Kellner will take part in an online Q&A session – no doubt this will form part of the conversation. I should emphasise that these are the personal views of Flockers – Mike Smithson)

What is it about their weightings?

For good reason, all polling firms apply weightings to their raw data in order to derive the final figures. There is little value in seeking to predict the responses of the electorate from the responses of a sample if that sample is not representative of the electorate as a whole; a poll taken in the Birkenhead working men’s club would have very little predictive value.

Looking at the YouGov data, I was immediately struck (see table above) by the resilience of the unweighted Conservative vote share and of the unweighted Conservative lead over Labour.

In each of the eleven 2010 polls to date, the unweighted Conservative share was in the range 41-46 (reported 38-40) and the Labour share in the range 26-30 (reported 30-33). The average unweighted Conservative lead was 14.4 points against an average reported lead of 8 points, indicating an average weighting of 6.4 points away from the Conservatives.

Crikey! If YouGov are habitually over-sampling Tory supporters, how massive must the unweighted Conservative lead have been when Labour support slumped in the aftermath of the expenses scandal?

Not so massive. In seven reported polls during May and June the unweighted Conservative share was in the range 38-44 (reported 37-43) and the Labour share in the range 21-26 (reported 21-27).

The average unweighted Conservative lead was 15.7 points against an average reported lead of 15.7 points – the weighting against the Conservatives was 0.012 points. The weighting moved in the Conservatives’ favour in three polls, against them in four.

When did YouGov’s weightings start causing the weighted result to diverge so dramatically from the unweighted result? Quite recently:

In 16 reported polls for which the fieldwork was completed in July, August or September the average unweighted Conservative lead was 14.4 points against an average reported lead of 12.8 points, indicating an average weighting of 1.6 points away from the Conservatives, far below the 2010 average weighting of 6.4 points.

The weighting moved in the Conservatives’ favour in four polls, against them in twelve. During October, November and early December, 15 polls gave the Conservatives an average unweighted lead of 15.5 points, against an average reported lead of 12.6 points, indicating an average weighting of 2.9 points away from the Conservatives.

In the twelve subsequent polls the average weighting against the Conservatives has been 6.3 points.

Every single poll in 2010 has seen the Tory share weighted downwards and the Labour share weighted upwards, with a net effect on each poll of between three and nine points. Ten of the eleven polls this year have seen weighting impact the Tory share by more than four points. Only five of the previous 39 polls was adjusted by the same amount.

The previously close correlation between the score produced by an analysis of the unweighted responses and the weighted results has been dislocated.

Why? If the weighting’s haven’t changed (and I understand that’s what YouGov will say), then the only answer can be sample bias. Have all eleven of YouGov’s 2010 samples been heavy with people having Conservative-leaning characteristics (but who, oddly, are not influencing the numbers saying they will vote Conservative)? YouGov does not weight by certainty to vote – which would be one credible explanation if this discrepancy was seen in another pollster’s work.

A historical perspective But surely none of this matters, because unweighted samples are irrelevant. Well, quite.

But what if I were to tell you that in six of YouGov’s last seven polls before the 2005 general election a simple analysis of the unweighted sample would bring you to within a 2% Lab overstatement of the actual result? Not bad for an unrepresentative sample. As it happened, the 2005 polls saw on average a two point weighting to the Conservatives, which made YouGov one of the most impressive performers in the 2005 general election.

By way of further historical perspective, a random sample of eight 2008 polls produced a mean weighting against the Conservatives of 2.3 points, and a random sample of five late 2007 polls produced a mean weighting against the Conservatives of 2.6 points. The current level of weighting appears to be unprecedented for YouGov in the Brown/Cameron era.

Good news for the blues? Not entirely. The unweighted samples support the recent narrowing of the polls, with the Conservative lead in the last three polls averaging 12.33 per cent, against an average 17.9 per cent lead in the first three polls of the year. Indeed the reported numbers may even have masked the extent of the narrowing, as the polls early in January were some of the most heavily weighted in the sample. The data also shows a steady increase in Labour support since May.

However, if you are prepared to look through the weighting, the basic unweighted responses of people polled by YouGov still point to a commanding Conservative lead.

Flockers