Why you should be wary of surveys taken during long weekends

With so much going on the political front there are almost certain to be opinion surveys carried out this weekend. If they are then treat them with extra caution because they might be subject to the bank holiday effect.

Many will remember the famous Populus tracker poll that was published on the day before the General Election last year and showed Labour on 41%, the Tories on 27% with the Lib Dems at 23%. While the figure for Charles Kennedy’s party was almost spot on the poll over-estimated Labour’s eventual support by five points and underestimated the Tories by six.

In the post-mortem afterwards Labour’s 14% lead was put down to the “bank holiday effect.” For whatever reason pollsters find it extraordinarily difficult getting balanced samples during long holiday weekends. Tories, in particular, are in very short supply.

The survey was based on interviews that took place on the Saturday, Sunday and Monday of the May bank holiday weekend. The tracker poll before that and the Populus final poll showed much smaller Labour margins.

Looking at the detailed data from the Populus 14% poll you can see the challenge the pollster faced. For every two people they could find who said they had voted Labour at the previous General Election less than one person was interviewed who had voted Tory.

The problem is that although correcting mechanisms are in place to deal with sample distortion not all is taken into account. Populus is currently weighting its polls on the basis of a 9% Labour lead from last May – triple the actual figure. I am much happier with the ICM calculation which has it at 6.5%. Still Populus is substantially better than Mori which has no past vote weighting adjustment at all.

This bank holiday phenomenon is so well-known that Anthony Wells of UK PollingReport suggested on Tuesday that the the April ICM survey for the Guardian had been put back a week to avoid interviews taking place over Easter.

So if a poll does come out during the next few days check the survey dates very closely. For any sign that Labour is weathering the current crisis might be because party supporters are much more likely than Tories to be be at home answering the phone during long weekends.

Mike Smithson