How predictive of the House Popular Vote is the Generic Ballot?

Question

The site FiveThirtyEight has an "updating estimate of the generic ballot, based on polls that ask people which party they would support in a congressional election". My question is predictive of the House Popular Vote is the Generic Ballot?

I'd imagine that even at the level of a single individual, the response of an individual to a generic ballot poll may not be predictive of how they'll vote in the actual election, because they may vote based on other characteristics of a congressional candidate other than party affiliation. But does these differences tend to cancel out at the level of the entire national electorate, so that Generic Ballot predicts the House Popular Vote really well? Or is it not very predictive? Have there been any studies on this?

Answer 1

TL;DR: historically, the average of the generic ballot polls taken the week before the election has been about as accurate as any poll. However, the polls themselves swing wildly, so the values may change over longer periods of time. I.e. the current polls aren't necessarily predictive of how the 2018 polls will be.

According to Real Clear Politics, the generic congressional ballot polling average was off by

• 1.7% towards the Democrats in 2016.
• 3.3% towards the Democrats in 2014.
• 1.4% towards the Republicans in 2012.
• 2.6% towards the Republicans in 2010.
• 1.7% towards the Republicans in 2008.
• 3.6% towards the Democrats in 2006.
• 2.6% towards the Democrats in 2004.
• 2.9% towards the Democrats in 2002.

A quick caveat. In 2013, the Democrats were up by 6.5% on October 29th. The final election result in 2014 was 5.7% for the Republicans. So there is at least one example of 12% swing over the course of a year.

Another example. On October 24th, 2016, the Democrats were up by 6.2% but on election day they lost by 1.1%. That's a 7% swing in just two weeks or so.

Given the caveat, they are reasonably accurate when taken just before the election. So when you read a generic ballot poll, remember first that individual polls are less accurate. Consider looking at a polling average. E.g. for 2018. Next, remember that what it is telling you is roughly where the election stands today, plus or minus 4%. The polls may change radically over the next year.

It also may be worth noting that the seat results may be different. As we saw in 2016, the districts that saw the biggest shifts away from the Republican candidates were the already Democratic districts. If that trend is still true for 2018, it may limit the changes somewhat.

The reason why I'm using the Real Clear Politics data is that they are nice enough to track both of the numbers needed on the same page. And, well, I read them regularly and thus know how to find that data. Other polling aggregators may give slightly different results.

Answer 2

FiveThirtyEight has actually discussed (several times) how predictive the generic ballots are for the US midterm. The answer is that they're pretty predictive, so long as you control for who's in the White House and allow for some regression to the mean; but they're not predictive in the sense that +1 in the generic ballot margin doesn't necessarily translate to a change of +1 in the eventual outcome.

From this 538 article, we see the average generic ballot in October of the year before the midterms, compared to the popular House vote (correlation coefficient of 0.90):

Even 1.5–2 years in advance, it's still pretty predictive (correlation coefficient of 0.78). From this 538 article:

Note, however, that a margin of +n in the generic Congressional ballots does not directly translate to a margin of +n in the popular vote. Between the early congressional polls and the election, there seems to be a sort of regression to a default state where the president's party loses by about 5 points, and all the deviations away from this "default" are smoothed out over the course of the year(s).

(Aside: the second of those 538 articles also does the same analysis as Brythan did in their answer concerning the generic ballot polls immediately before the midterms, and comes to much the same conclusion.)