Why those opposed to Brexit shouldn’t get too excited by the BMG 10% Remain lead poll
Ex-YouGov President, Peter Kellner, says be cautious
The former President of YouGov and leading political commentator, Peter Kellner, has written a comprehensive note about the BMG poll for the Indy which has Remain 10% ahead to a question of how those sampled would vote in a future referendum. In his look at the BMG numbers Kellner notes:
“First, they are mainly driven by a seemingly huge shift in people who did not vote in last year’s referendum. Sure, the Remain camp is being swelled by young adults who abstained last time or were not old enough to vote. But previous BMG polls included this phenomenon. During the summer, when previous non-voters were asked how they would vote in a fresh referendum, they divided fairly steadily: around 45% Remain, 25% Leave. Now, suddenly, BMG say the divide is 67%- 16%. As this group comprises more than a fifth of BMG’s total weighted sample (299 out of 1,363), this 51-point Remain lead within this group accounts for the whole of the reported overall Remain lead.
However, BMG did not actually interview 299 previous non-voters. Its unweighted subsample was barely half that: 156. The margin of error on such a subsample is large – and the very fact that BMG could not track down as many non-voters as it wanted, provides a clear warning (as I know from my experience at YouGov) that the sub-sample may not be as representative as one would like.
Secondly, if it were true that, despite these sampling issues, there had been a sharp shift among non-voters to Remain, there would be some echo of this among other groups. In particular, one would expect to see clear signs of growing buyers’ remorse among Leave voters. That wouldn’t apply to hardline anti-EU voters – something like two thirds of those who voted Leave 18 months ago. But it would be likely to have some effect on the one-third who were “instrumental” voters: people who are not viscerally anti-EU, but believed that Brexit held the best hope of more jobs, higher pay, less crime, a better-funded NHS and improved access to public services such as local schools and social housing.
The point is that any weakening of these pro-Brexit arguments that is liable to shift large numbers of non-voters into the Remain camp, should also produce some shift among the “instrumental” Leave voters. But BMG’s figures produce no evidence of this. There is no statistically significant rise in buyers’ remorse among Leave voters in this poll compared with previous BMG polls.
The third reason to doubt that BMG’s figures reflect a public reaction to last weeks’ Government defeat in the House of Commons is that the poll was conducted before the vote. Its fieldwork dates were December 5-8 – that is the week before last.
Fourthly, other polls within the past fortnight show no indication of a lurch to Remain. Two YouGov surveys and one ICM poll indicate that nothing much has changed in recent weeks, despite the turbulence surrounding the events leading to the EU’s decision to allow Brexit negotiations to move to stage two. They are consistent with a narrow Remain lead, compared with a narrow Leave lead before this year’s general election. Over the past six months, as I discussed in a recent blog, there has been a small shift to Remain, but only a small shift.
To say this is not in any way to denigrate the quality of BMG’s work. Small subsamples of hard-to-reach groups must always be examined with care, for they are liable to trip up even the best research companies. BMG seem to have been unlucky, not culpable.
The larger point is one that applies to enthusiasts on any side of any issue. It is important to resist the temptation to cherry-pick those polls and findings that support one’s case, and ignore those that don’t. Remember the warning of those great 20th century philosophers, Simon and Garfunkel, in The Boxer: “All lies and jest: a man hears what he wants to hear, and disregards the rest”.
Like all shock polls we need to see if other surveys from other organisations show the same broad trend. So at the moment we must wait. As Peter writes it is too easy for people to believe what they want to believe.