When following the polls and the markets can be expensive?

If things had been going to plan I would have spent yesterday in Manchester with the Sky News team acting as one of Adam Boulton’s pundits at the Labour leadership conference. Thankfully circumstances prevented the trip because otherwise I might have been associated with the TV station’s wrong call, along with the Daily Telegraph, that Alan Johnson had won.

For it’s all too easy to get carried away with what betting markets are pointing to and to assume that those risking their money must somehow have better information.

For the Labour Deputy Contest of 2007 follows the Dunfermline by-election of 2006 and the IOC’s London Olympic city election of 2005 when super-firm odds on favourites did not make it.

A major factor here was that to its great credit the Labour movement carried out the complex counting operation in complete secrecy and there were no leaks until the contenders had been told and arrived in the hall.

The reasons that many went for Alan Johnson were the YouGov polls of Labour members and trade unionists that were finalised on May 28th and appeared in the Sunday Times on June 3rd. This is how these compared with what actually happened – (figures from Luke’s blog)

Party Members:
Johnson 24% on YouGov, 17% in vote
Benn 24% on YouGov, 22% in vote
Harman 17% on YouGov, 24% in vote
Hain 13% on YouGov, 12% in vote
Cruddas 12% on YouGov, 17% in vote
Blears 9% on YouGov, 9% in vote

Trade Unionists
Johnson 26% on YouGov, 14% in vote
Benn 22% on YouGov, 15% in vote
Harman 16% on YouGov, 13% in vote
Hain 15% on YouGov, 21% in vote
Cruddas 15% on YouGov, 27% in vote
Blears 7% on YouGov, 11% in vote

What’s clear here is that in both surveys there was a massive over-statement of Alan Johnson’s support and an understatement of John Cruddas. The latter is mostly explained by the timing of the fieldwork which ended before the big Newsnight debate when Cruddas came over very well. The former is a problem and I look forward to hearing what YouGov’s Peter Kellner has to say.

It was that poll, it will be recalled, that prompted the big movement to Johnson in the betting. From that point on he became favourite and his price got tighter and tighter right until the results were announced officially. This, no doubt, had a big influence on the media coverage.

For the three preceding months the betting had been driven by two earlier YouGov polls which had Hilary Benn on 36% well ahead of Johnson.

With such a complicated election process using the AV system this was always going to challenging for the pollster that built up a great reputation in such contests by getting both the 2001 and 2005 Tory contests right to within 1%. In those fights the final surveys took place after most members had returned their ballot slips. In the Labour deputy race the final poll was actually quite old.

My betting shift from Benn to Harman at 8.8/1 on June 1st was driven by a confident prediction on transferred votes from a friend who is a member of the Harman team and the YouGov finding that I reproduced in this post. This showed very strong support for the notion of a woman being there.

Mike Smithson