The limitations of polling: How Americans responded when asked if a law that didn’t exist should be repealed
What happened when YouGov asked Americans whether a fictional 1975 Act should be repealed. huff.to/Zcb4dD twitter.com/MSmithsonPB/st…
— Mike Smithson (@MSmithsonPB) April 12, 2013
An experiment by YouGov in the US for HuffPost
The above charts are taken from an excellent piece by Mark Blumenthal of Huffington Post on polling about specific issues.
When phone samples were asked for their views about the repeal of the 1975 Public Affairs Act, a piece of legislation that doesn’t exist, 20-40% of those questioned were ready to offer views even though this was entirely fictional.
The test was repeated by YouGov in an online survey for Huffington Post in which far fewer people were ready to offer an opinion – but even so 9% said repeal and 6% disagreed.
Then as a further test separate samples were asked for their views with a partisan element included in the question.
One group was told that “President Obama says the 1975 Public Affairs Act should be repealed” while another was told that “Republicans say” it should be repealed. The findings are in the chart above which shows how partisan respondents were.
The test also included questions about what the fictional act actually did and a section of the sample were ready to offer opinions.
Blumenthal concludes: “The lesson here is straightforward: Many poll respondents will offer opinions on issues they know little or nothing about, making it difficult to distinguish pre-existing opinions from reactions formed on the basis of the words of the question. Poll respondents will find it even easier to offer an opinion when informed where well-known political leaders stand on the issue.”
There’s a great lesson for us all in this test.
Mike Smithson
For the latest polling and political betting news