Why you should be wary of hypothetical polling
Please indulge me as I take a minute today to complain about poll questions that ask people whether they'd be more or less likely to vote for a candidate if the candidate did X thing.
— Ariel Edwards-Levy (@aedwardslevy) August 29, 2018
Things people are generally bad at:
-explaining how they decide their votes
-entertaining counterfactuals
-understanding probabilityThis asks them to do all three!
— Ariel Edwards-Levy (@aedwardslevy) August 29, 2018
But the even bigger problem is that people aren't going to be answering the question as asked.
Like, say you find a person who's a die-hard Democrat. They still have Clinton bumper stickers. The chance they're voting for Trump again is zero.
— Ariel Edwards-Levy (@aedwardslevy) August 29, 2018
A pollster calls them up, and asks how…I don't know, Trump's hypothetical support for a bill to legalize kicking puppies would affect their vote.
— Ariel Edwards-Levy (@aedwardslevy) August 29, 2018
Now….they weren't going to vote for him before, and they won't if he signs the law, so really it won't affect their vote at all. But that's not what they'll say.
— Ariel Edwards-Levy (@aedwardslevy) August 29, 2018
They'll say it makes you less likely to vote for him, either to register their dislike of Trump, or their dislike of puppy-kicking, or both.
— Ariel Edwards-Levy (@aedwardslevy) August 29, 2018
Still my favorite dumb case-in-point example of this… pic.twitter.com/b3HaFlUmbp
— Ariel Edwards-Levy (@aedwardslevy) August 29, 2018
It's not that these questions can't be useful, especially if there are others for comparison, but I think it's generally a mistake to take the results at absolute face value.
Anyways, back to your regularly scheduled programming!
— Ariel Edwards-Levy (@aedwardslevy) August 29, 2018
The tweet thread above by the HuffPo’s Polling Editor is very useful when we see this type of polling.
It is the second tweet of this thread that is particularly noteworthy, so the next time you see a poll about voters being more or less likely to vote for a candidate if the candidate did X thing remember this thread.
In the next few days I’ll do a fuller a thread on why for similar reasons polling asking what the voting intentions would be if parties were led by X,Y, or Z are similarly flawed.