Beating a Dead Horse: Election Polling is Flawed

Just this week, we got another pseudo apologetic letter about why election polling is sometimes (perhaps even often) inaccurate. This time it was from Peter Loewen (an Assistant Professor of UofT). His article in the Ottawa Citizen is here but, it like many similiar pieces completely misses the point.

Pollsters are in a unique an perilous position in the days leading up to an election because as much as they want o believe it it not true; voters are allowed to change their mind.

Peter’s article which is largely about the Quebec provincial election is a disservice to the industry. His focus is on the problem of getting a good sample. In his own words:

Pollsters simply do not know enough about who responds to polls via some media, who replies through others, and what kinds of people ignore polling requests entirely. The problem isn’t getting sample, it’s getting good sample. (Peter Loewen)

Does sample matter? Yes. Is their a challenge with getting a representative sample given response rates and new technology? Yes. These are not even close to being the issues that challenge pollsters in being more accurate about the vote. They are simplistic and naive assertions which miss the fundamental challenges of election polling.

1. Not Everyone Votes

Public opinion polls at there best are uncomplicated because everyone has an opinion or a non opinion. When it comes to elections, what people say is their vote choice (or non choice) is a spur of the moment decision that is divorced from actually voting. While pollsters can expect that most people who express an opinion will actually vote, we know that there is a gap when 75% give us a vote preference but only 50% or so turnout. What we lack is a guaranteed method of identifying those who will actually vote. The best sample in the world won’t address the gap between those who have a preference and those who will vote.

2. Voters can Change their Mind

I am sure that many would prefer that people did not change their mind but they do. And, given the importance of a democratic franchise, perhaps we should applaud those ballot box reconsiderations. In either case, voters are allowed to be coy — about their intention — and ultimately deceive pollsters. Good pollsters and good media know when this is a possibility.

So, sure, the sample matters but in reality, it could easily be argued that many of failures of polling are simply a reflection of a misguided focus on predicting a future election outcomes (the result) and not enough focus on the meaning inherent in what people tell us. There is very little science that can tell us how people (rather than atoms) will behave in a particular situation. Perhaps it is time to stop expecting polling to do that.


, ,

About Richard Jenkins

Market research professional and small business owner

View all posts by Richard Jenkins


Subscribe to our RSS feed and social profiles to receive updates.

4 Comments on “Beating a Dead Horse: Election Polling is Flawed”

  1. Ellis Westwood (@elliswestwood) Says:

    Great perspectives, Richard.

    I think Loewen would agree with some of your points. He reminds us that elections “are about changing minds” so of course we would see volatility in polling results during the course of an election. I also agree that voters made last-minute decisions in this election, perhaps more so that in other provincial elections in the past year.

    While you both start with different premises, perhaps you and Loewen would both agree that polling during elections serves little purpose because they cannot an accurate enough prediction of the final outcome – for a variety of methodological reasons.

    Instead, polls may simply serve as material for media to cover elections as though they were horse races – who’s up, who’s down, who’s ahead by a length – and as entertainment rather than serious research instruments.


  2. John Says:

    If the polls show Obama pulling ahead, the markets will collapse just as they did in the fall of 2008 when it became obvious McCain was going to lose. The smart money quickly moved their money elsewhere. The FED has been printing money ever since to make up for that capital flight. It’ll be far worse this time because a lot more people “get it”.


  3. Peter loewen Says:

    Hi Richard,

    I am just seeing this now (but conveniently after the BC election). A quick thought: if it’s a fact that voters change their minds during elections, and if it’s widely known by pollsters, then why don’t the public ones just model this? Surely, if everyone knows what happens to the trend them this can just be put into late campaign polls, no?

    It’s easy to say that pollsters should focus on other things – like what is causing vote choice. I agree. But they don’t, and no amount of messenger shooting will change that.

    Like the. Log,
    Peter loewen


  4. Peter loewen Says:

    Like the blog is what I meant.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: