Why were pre-election polls and forecast models inconsistent with the election of Donald Trump?

Question

As it's obvious now, pre-election polling and forecast models were inconsistent with the vote count in this election cycle as not many predicted a Trump victory.

Is there any particular reason why polling was so off?

Answer 1

Reasonable theories I have heard have included:

• A change in polling foundations (home phones become cell phones = limitations on traditional cold calling... and also the shift into online polls). Plus perhaps the diminishing patience people have with enduring the polling process (I believe the percentage of people who agree to it has dropped consistently)

• A hesitation/fear that keeps some people from admitting that they support a candidate who is being widely presented as detestable/deplorable in much of the public discussion/media. [apparently an effect common enough to have a term, see the Shy Tory Factor... the possibility it might be important was even presciently suggested in a question here a couple weeks ago]

• A possible tendency for people to publicly show full support for minority/heroic candidates because it's the socially favorable thing to do, even when they're actually entertaining uncertainty internally. [this is along the lines of the Bradley Effect]

• People failing to turn out as they've indicated they will... perhaps due to weather (though it's fairly unlikely this was a big factor in this election), a false sense of security (such as when the polls in the days leading up to and into election day suggest a comfortable victory!), or just a general failure to muster the effort/will to follow through with the voting.

Regardless, there is a widespread trend of polls failing recently, and maybe even specifically underestimating the conservative side. In 2015, the UK Parliament was projected to come out about even between the top two parties, but the conservatives won by over 5%, ending up with the majority (which was considered to be a near 0% chance possibility as the day began), this summer Brexit passed (considered almost certain to fail as the day began, ended up passing by 4%), and last month the Colombian Peace Referendum failed (after being consistently polled to pass by about 10%). So perhaps this is a trend to be aware of going forward until polling methods can hopefully adapt. Others here have also pointed out there was an underforecast conservative swing in the 2016 Iceland election and changes in the Swedish election.

Note, though, that most models showing the full spread of possibilities didn't say this was a set Clinton win, but that it leaned maybe 70-80% likely that Clinton would win.
A 1 out of 5 chance of being wrong is not insignificant...
If they say there's a 20% chance of rain today, you shouldn't be surprised if it does rain! In meteorology we've often got the same continuing struggles, particularly when it comes to issues like hurricane/storm forecasting; getting people to understand the uncertainty and full potentials of realistic possibilities. This is something we should keep on our media to better portray to us, and something school math courses could better focus on, perhaps being of significant benefit to a great many.

Considering that most polls have maybe a 4-6% typical error, this result was really quite within the range of possibility for most (although their consistent bias suggests that there really are some fundamental shortcomings). But, still, most of the quality election forecasters did also measure in some considerations of this trend towards less poll reliability, and were urging caution regarding overconfidence in the indicated spread (as Nate Silver of fivethirtyeight did here on election morning).

Answer 2

Two points:

First is that the system of electoral college or first past the post (winner gets it all) magnifies the differences in the case of close calls. Had 80.000 votes change sides in Florida and 40.000 in Pennsylvania, and Hillary Clinton would be POTUS.

Also, the electoral college system turns one very big poll into 51 smaller polls, each one necessarily with a bigger error margin (due to the reduced sampling data).

The popular votes for each candidate is a better guideline about the validity of polls, and there the polls have not failed by that much.

The second is that poll organizations do not simply interview people, get the mean value and publish it as their predictions. It is common knowledge that this method is not accurate enough.

What they do is to build models that try to explain the relationships between the (historical) poll data with the actual results, trying to identify and weight the different effects that can affect the final result: Bradley effect, "Shy Tory" effect, race and sex of the candidates, effects of the economic cycle, even the effect of weather.

And of course, as any model based in historical data, its effectivity relies in that the situation in the current electoral cycle is not very different from the situation in the previous electorals cycles. They may account on some Trump supporters hiding their vote intention because of peer pressure/embarrassment using historical data, but if peer pressure is higher in this election than in previous elections it is difficult to tune up that effect correctly (how can you measure peer pressure to adjust your model before the election day?).

This electoral campaign has added a establishment/anti-establishment dimension and has been specially polemic, so the historical data has been less helpful.

Answer 3

First, you should ask yourself if the polls were wrong. The latest Real Clear Politics average gave Hillary Clinton a 3% lead over Donald Trump. It looks like she is going to win the popular vote. So the polls were only off by 3%. They were also off by 3% in 2012, just in the other direction. Polls are imperfect by nature. A 3% miss is a normal occurrence.

Update: the first paragraph was written immediately after the election. The initial narrow win of the popular vote turned into a 2% win. So the national polls were only off by 1.2%. They were more accurate in 2016 than in 2012. However, since the electoral college is not directly affected by the national vote margin, they were measuring the wrong thing. Running up the score in California was not helpful.

That said, there is a reason to think that there might have been skew or bias in the polls.

Demographics

United States presidential polls usually adjust their results to match the demographics of the most recent presidential election. In 2012, this caused the polls to miss demographic changes which created a 3% polling miss. In 2016, it seems likely that the polls missed a decrease in the African-American vote and an increase in the working white vote. Also, millennials may have stayed home without an inspirational candidate like Barack Obama or Bernie Sanders.

Nationally this only had a modest impact (1.2%), but in a few crucial states, this may have swung the vote more than expected. Trump won Wisconsin, Michigan, and Pennsylvania, three states that are normally considered safe for the Democrat. Also three states with an exceptionally large white working class vote, which underperformed in 2012. And they have very few Hispanics to be offended by Trump's immigration stances.

We'll probably start seeing numbers that better explain what happened over the next several days. Last night people were trying to project the current vote totals. It will take a day or two for them to recover and get to work on the differences between the polls and the actual results.

Update: and we did see that. The actual polling miss nationally was only 1.2%. The largest state polling miss was 7.2% in Wisconsin, but in fifty states, we'd expect two or three states to fall outside the margin of error. It's a 95% confidence interval. The polling miss in California was almost as large at 6.5% and affected more people (6.5% of people voting in California is more than 7.2% of people voting in Wisconsin). But no one seems interested in commenting on that. Or on the 9% miss in 2012.

Answer 4

Polling relies on answers of people and people can lie or hide the truth.

Normally speaking, those who are very proud of their vote will be eager to tell who they are going to vote. On the other side, those who are aware of their vote not being "conventional" tend to hide it and do not show in polls.

This has happened recently in different referendums: Brexit in UK and Peace agreement in Colombia, where polls where pretty sure about one direction of the vote and all this hidden vote ended up making the difference.

The bottom line of this is not the fact that people consistently lie. To my impression, polls also create an ambiance of possible results: if you see that "your" candidate is going to win by a big margin and you are not superfan of him/her, you may just relax and skip voting. On the contrary, if "your" candidate is losing in the polls you may consider your vote more important to omit and can even flirt with voting for someone despite not supporting him/her completely (let's call it a protest vote, see French presidential election in 2002, with a lot of people voting for small groups on the 1st round).

Let's put it with an example: you support one club in a given sport. This Saturday there is a game versus a weaker team. Since there is little certainty about who the winner will be (most likely, your team), you may relax and skip watching the match because nothing important can happen. On the contrary, if you are playing versus a team on your level, chances are that the match will be quite equal, so you will do your best to go and watch it.

To me, the same applies to elections. The examples on UK, Colombia, Sweden and Iceland (see comments) are all good examples of this. I call this demobilization due to expected result.

Answer 5

I feel like there were at least two main flaws with the polls:

1. Some polls were asking the same group of people questions on which candidate they support, but they should have been sampling randomly selected voters each time (this article mentions this).
2. They didn't take into consideration the "closet Trump supporters." For some people it's social suicide to publicly announce that you are voting for Trump (or they feel hesitant to reveal this info to pollers over the phone). He's been painted as a misogynist, racist, xenophobe, homophobic, etc. and it's obviously unpopular to support someone who is (or at least appears to be). Not to say that Trump supporters support those things (I'm sure some do), but they saw him as a better alternative to Hillary, and could overlook his flaws so that he could get the job done.

Answer 6

As also mentioned in the other answers, the turnout was a major factor here. One has to consider here the nature of the contest: Large low density rural areas with a majority of Trump supporters vs. a few concentrated areas where the vast majority are Clinton supporters. So, even though the popular vote nationwide is pretty close to 50-50, locally you have lopsided contests where Trump can get, say, 70% of the vote from rural areas, while Clinton could get 70% of the vote from inner cities.

A lower than expected turnout in the inner cities, combined with less lopsided margins for Clinton there can easily lead to a wrong prediction of the result of an entire State. E.g. Michigan was expected to be solidly in Clinton's basket, but in places like Detroit the Clinton vote was less huge than expected, leading to Trump winning that State.

Answer 7

I think there are several components. One that hasn't been mentioned before: possible deliberate manipulation conducted by media. Saying some candidate has fewer chance will discourage their voters to go and vote and would make undecided ones to likely vote his opponent.

Answer 8

Why where thy wrong? Simple, for once the media was extremely biased for Clinton, wrote her up as much as they could in the hope that people, that where against her would simply give up and so make it a self fulfilling prophecy. For example they published polls, that only asked people that already had voted in previous elections, and these where mostly Clinton, Trump got most of his votes from people that didn't vote before, regardless of reason.

They also overestimate their power and the power of the establishment and underestimate the power of the internet, in their world they still have ultimate power, to decide who wins and who not, but the internet works on different rules and remembers things, like Clinton's history, the network community brings these up again and inform people.

In another part they simply thought that the "blue dogs" (Democratic voters that always voted Democrats), would all vote for Hillary (as well as she herself thought that) but after what she pulled of against Sanders many of them just didn't vote at all or voted Trump just to pay Hillary back.

In the end the establishment tripped over its own arrogance, in the same way the British did with "Brexit"

Answer 9

I can give a different perspective/theory as to why polls fail, as I have seen many polls fail in India.

1. The poll includes very few people.

   Even with all the techniques to spread the polling population among masses, it still does not change the fact that most people may have different opinion than those who are polled.

2. Denial of fact

   Most media channels want to show what people want to see, not what they should be showing. For example, in the above case, US media seemed to be in a denial mode that, Trump had managed to radicalize the general public of US against muslims/refugees.

   The media kept saying that the public of US won't support a man who has islamophobia. But the fact on ground is that people wanted solution to the increasing attacks in US. They were ready to accept any solution, however weird/morally wrong the solution might be.

   Add to this that the people of US had not experienced any major terrorist attack for many years, it was easier to radicalize them using even small scale attacks, which trump did successfully. But media kept denying this. They didn't wanted to accept this. So they kept showing that Hillary would win.

3. Examples from India

   Same things happened here as well. Media said that Narendra Modi would never become PM, as he was blamed for Gujrat riots. But the result was that he got elected with one of the biggest victory in history. The reality was that all that people cared about was development, and modi seemed to be capable to deliver it. Nothing else mattered. Same happened with Kejriwal when he was elected CM of national capital Delhi.

Answer 10

I was listening to the very respected Charles Franklin from Marquette University, who runs their polls, talking about the polling vs the results on the radio on Thursday. His take:

The problems with finding people and getting a representative sample are well known issues and ones that pollsters can make statistical adjustments for. They can very accurately take the pulse of the general population.

What is extremely difficult to pin down and, according to him, the greatest challenge for accurate polling is determining who is actually likely to vote in any given election, both among the respondents, and how that maps out to the larger population.

Answer 11

I have two theories in place:

1) Embarrassment

One candidate is controversial / unpopular. Respondents are unlikely to admit to a surveyor about their true preference. At a voting station, there is privacy and true preference will take place.

2) Determination

Supporters of the side that is slightly behind are more motivated to turnout and vote. As opposed to the leading side that may grow overconfident.

---

I'm not a doctor, not a lawyer, not an investment advisors, just some theories, common sense and critical thinking.

EDIT / UPDATE: I wrote about two matters that I've discussed with my wife. No sources, no references, just pure intentions and sharing our experiences. It could be improved... I could spent endless research, references, sources... I'm at the source, one of my domains is "mostly doing" because I've learnt that talking about stuff doesn't move the needle.

Answer 12

Some published conclusions from a 2016 paper signed off by dozen [or so] researchers/pollsters [of course, alway too late to get much votes here]:

There are a number of reasons as to why polls under-estimated support for Trump. The explanations for which we found the most evidence are:

• Real change in vote preference during the final week or so of the campaign. [...]

• Adjusting for over-representation of college graduates was critical, but many polls did not do it. [...]

• Some Trump voters who participated in pre-election polls did not reveal themselves as Trump voters until after the election, and they outnumbered late-revealing Clinton voters. This finding could be attributable to either late deciding or misreporting (the socalled Shy Trump effect) in the pre-election polls. A number of other tests for the Shy Trump theory yielded no evidence to support it.

Less compelling evidence points to other factors that may have contributed to under-estimating Trump’s support:

• Change in turnout between 2012 and 2016 is also a likely culprit, but the best data sources for examining that have not yet been released. [...]
• Ballot order effects may have played a role in some state contests, but they do not go far in explaining the polling errors. [...]

There is no consistent partisan favoritism in recent U.S. polling. In 2016 national and statelevel polls tended to under-estimate support for Trump, the Republican nominee. In 2000 and 2012, however, general election polls clearly tended to under-estimate support for the Democratic presidential candidates. The trend lines for both national polls and state-level polls show that – for any given election – whether the polls tend to miss in the Republican direction or the Democratic direction is tantamount to a coin flip.

[...]

A proposal for addressing the performance of state-level polling. As this report documents, the national polls in 2016 were quite accurate, while polls in key battleground states showed some large, problematic errors. It is a persistent frustration within polling and the larger survey research community that the profession is judged based on how these often under-budgeted state polls perform relative to the election outcome. The industry cannot realistically change how it is judged, but it can make an improvement to the polling landscape, at least in theory. AAPOR does not have the resources to finance a series of high quality state-level polls in presidential elections, but it might consider attempting to organize financing for such an effort. Errors in state polls like those observed in 2016 are not uncommon. With shrinking budgets at news outlets to finance polling, there is no reason to believe that this problem is going to fix itself. Collectively, well-resourced survey organizations might have enough common interest in financing some high quality state-level polls so as to reduce the likelihood of another black eye for the profession.

Answer 13

A sinister explanation which may never be able to be proven. The 2016 election may differ from polling results because a decisive numbers eligible voters were:

• turned away

• incorrectly blacklisted by overly fuzzy implementations of the Interstate Voter Registration Crosscheck Program, (supposedly a fraud preventative database cleaner for purging or flagging duplicate entries from state voter rolls), which fail to require mandatory Social Security Number and birth-date comparisons given two voters named John Doe,

• only got "provisional" ballots

• never got to vote because there were too few early voting places, or too few voting places to process the volume of election day voters.

There are a number of articles on how gutting the Voting Rights Act of 1965 might explain why the polls and election results seem to contradict each other. Journalist Greg Palast maintains various voter suppression tactics lead to the election of Bush as well in 2000.

Answer 14

The question assumes the polls and forecasts were actually wrong, as do most of the answers so far. Yet it's also not inconceivable that the polls were only apparently wrong, (i.e. those polls were correct), as would be the case with user chx's voter suppression answer, and would be logically consistent with the 45th President's historically low approval average, (currently 39.1% as of 8/5/18).

Other possible ways in which polls might only be apparently wrong:

• Insufficient voting machine build quality. During manufacture or maintenance, some necessarily durable mechanical or electronic part is replaced with a substandard part to save a few pennies or dollars, and thus help balance some quarterly departmental budget. The substandard part eventually fails sporadically, perhaps failing to process input as designed. If the price of maintenance calls and proprietary machine parts are high, cash poor precincts might be more liable to delay necessary maintenance, e.g.: neglecting to promptly replace a dried-out rubber roller that occasionally fails to grip a paper ballot in an optical scan box, etc.

  Cash poor districts usually vote for different candidates than wealthy districts, so the effect of delayed maintenance would be unequal and disproportionally harm the poor. The impact of a system bug's output might be symmetrical, random, lopsided, or strangely specific -- but bug reporting might be better in wealthy districts with larger budgets for operator training, in which case poorer districts would again be hurt more.

• Voting machine software, firmware, hardware, or design bugs. Some bugs might result in output errors leading afterwards to an appearance of incorrect polling results. While virtually all such complex systems have bugs, few computer systems inspire such persevering faith in their reliability as voting machines grace their keepers with.

• Exploits, hacks, etc. based on those bugs and design flaws. There has always been motive; some of the known voting machine flaws already furnish plausible methods, (to say nothing of what zero day exploits may exist); the opportunities required might depend on either the methods, or else on old-fashioned human carelessness or corruption.