5

I've always wondered about the type of people who take surveys. Who among us would take a random call in the middle of the day from a perfect stranger and spending the next 30 minutes answering dozens of extremely intrusive questions about your personal life and political beliefs?

According to Pew, survey response rates have fallen precipitously in recent years to a new low of 6%: enter image description here

From my research, I noticed that a key difference between Liberals and Conservatives is how trusting they are of strangers

My thinking is that if a person is less trusting of others, then they are more likely to use a call blocker on their phone and an adblocker on the Internet. If a stranger does manage to contact them, they are less likely to have a 30 minute chat about all of their most deeply held beliefs.

Indeed, a study recently found that 11.7% of Republicans admitted they would not give their true opinion to a pollster vs 5.4% of Democrats. The top 4 cited reasons were:

  1. A lack of trust in phone polls as truly being anonymous.

  2. An apprehension to associate their phone numbers with recorded responses.

  3. Fear that their responses will become public in some manner.

  4. Fear of reprisal and related detrimental impact to their financial, social, and family lives should their political opinions become publicly known.

These reasons would also apply to online surveys as well, which would explain why there's no significant difference between the results of online and phone surveys. For example, Pew Research will actually contact people in real life (via snail mail or otherwise) and invite them to participate in online surveys.

If the research is correct that Conservatives are less trusting of strangers, does this mean they are less likely to answer (or at least give truthful answers to surveys) and are thus under-sampled by polling institutions? Do pollsters correct for this error?

3
  • 2
    Lying to an interviewer certainly is an issue, but the graph does not show that. What the graph shows is that people have become less and less likely to answer phone calls from unknown numbers thanks to caller ID. Oct 31 '20 at 16:24
  • 1
    I see refusing to answer a survey and refusing to give correct answers on a survey as two sides of the same coin coming from a distrust of pollsters. They could both exert a systematic error on the final results of polling so this question addresses both issues together. Oct 31 '20 at 16:36
  • 3
    how can one take a poll about how honest folks are to pollsters? Seems a little recursive... How's that conversation go; "well you guys are cool, but I lie to the other pollsters"...
    – dandavis
    Oct 31 '20 at 19:34
2

While pollsters do worry about the "shy voter" problem (people who lie about their political preferences), they do not correct for it. The shy voter problem may have been a factor in the 2016 election, but it was not the reason that pollsters failed to predict Donald Trump's election. Polling did properly predict the Hilary Clinton would win the popular vote. Having a plurality of the popular vote doesn't mean anything when it comes to presidential elections. Winning the Electoral College vote is all that matters. Where pollsters did fail in 2016 was in a proper prediction regarding who would vote in swing states, and in particular, in Pennsylvania, Michigan, and Wisconsin.

There is a concern once again in 2020 of the shy voter problem, but this time around there is also a concern that there may be shy Biden voters. An even greater concern is the huge early voting and absentee voting in 2020. As an example, more people have already voted in Texas in 2020, three days prior to Election Day, than the total number of Texans who voted in the 2016 Presidential Election. The unknown voter problem is widely perceived to be a much more significant issue than is the shy voter problem.

6
  • 2
    I don't understand the claims of shy biden voters. You don't see people protesting in front of Biden supporters homes, assaulting Biden supporters in the street, or hunting them down and shooting them. There isn't a social cost to publicly supporing Biden. There IS a cost to publicly supporting Trump.
    – Ryan_L
    Nov 1 '20 at 4:38
  • @Ryan_L There is a cost to publicly opposing Trump in some parts of the country. I live in one of the most conservative areas in Texas. I get grief from some of my neighbors for not having a Trump sign in my yard. While I am not a liberal, I am a never-Trumper, and my neighbors know that. Some of those neighbors, people who nominally would rather die than vote for a Democrat, have told me they voted early for Biden, but they also told me not to let anyone know that. They are shy Biden voters. Nov 1 '20 at 4:56
  • 1
    I'm sure some exist, but it seems extremely implausible to me for them to be anywhere near as common as shy Trump voters. You must see the difference in consequences each would face. In Portland, there was a Trump supporter stalked and killed while just walking down the street. I don't see that kind of thing happening to Biden supporters.
    – Ryan_L
    Nov 1 '20 at 5:05
  • @Ryan_L Look back to 2016. Suppose you were in Pennsylvania, Michigan, or Wisconsin. Everywhere you went you would see yard signs for Clinton. Your relatives, friends, and neighbors all told you they were voting for Clinton. But suppose you secretly didn't like Clinton, let alone what she represented. Lying to your close associates is perhaps reprehensible. Lying to a pollster? No problem. Nov 1 '20 at 5:34
  • 1
    I agree, but there is less incentive for Biden supporters to lie, because the consequences of being a known Biden supporter in a Trump supporting area are much less extreme. No Biden supporters have been stalked and murdered for wearing a Biden hat in a pro-Trump area. But that's what happened to Trump supporter Aaron Danielson in Portland. Biden supporters have to fear some arguments. Trump supporters have to fear some arguments and murder. All I am saying is that there is a difference in dangers each face, so we should expect a difference in the rates of lying as well.
    – Ryan_L
    Nov 1 '20 at 6:43
1

Survey research on a mass scale has never had great response rates. Trust is certainly part of it, but the main obstacle is the 'foot in the door' moment: convincing people to give researchers an opening to ask questions. The pace of the modern world is increasingly more rapid and pressured, and so (increasingly) people do not want to set aside the time to deal with a researcher, even if it's only minutes. There is too much to do in a day already, and too many easily accessible and enjoyable ways of filling idle time. Survey research is unlikely to be prioritized, unless someone has curiosity about the process or a sense of civic duty that makes them open to the questions.

Once researchers have their foot in the door (so to speak), they have a number of tactics which help them get honest responses. The primary tactic, of course, is institutional reputation. Typically, when someone calls and identifies themselves as a member of a well-known organization — "Hello, Im calling from Pew Research..." — it establishes a level of trust by leveraging the good name of the institution. Unfortunately, since the mid-2000s the more extreme elements of the US Conservative movement have put extraordinary effort into delegitimizing academic, scientific, and political norms and institutions (which did not serve their political agendas). The result is that people on the far Right suffer a trust-gap. I'm sure you've heard people refer to the Right-wing media bubble; that bubble is an expression of that trust-gap, where far-Right conservatives have drawn strong partisan lines about which institution are credible and which are not, a line constantly expanded and reinforced over the last four years by Trump himself (e.g., his 'fake news' and 'lame-stream media' type comments). That by itself should account for most of the 'trust' differences between Republicans and Democrats, since the latter tend to reject that delegitimizing narrative.

I don't think this would lead to conservatives being more inclined to answer dishonestly. Instead, it would lead to lower response rates among conservatives in proportion to the extremity of their positions. Dishonest answers come in two varieties:

  • Emotional repression (shame, anger, fear, etc), that makes someone instinctively shy away from the truth
  • Active disinformation, in which people intentionally lie in order to confuse the issue

Polling researchers have spent a lot of time thinking through the first problem, and have a number of tools to deal with such these emotional reactions. Things one might not think about — tone of voice, speech cadence, word choice, occasional levity, reassurances about the importance of the research — all help ease people's internal stresses and get them to open up. Active disinformation is (and always will be) rare, because active disinformation requires a kind of double-think that most people are not inclined towards: i.e., one must think about one's 'true' response, then actively decide to give a 'false' response for appropriate questions. Double-think is labor intensive, and only the most dedicated partisans would consider engaging it. It's much easier to not answer.

3
  • "it would lead to lower response rates among conservatives in proportion to the extremity of their positions" - Do you think this means there is a systematic polling error that biases the polling average away from the actual election result? Oct 31 '20 at 17:27
  • @SurpriseDog: Any polling institution worth its salt has sophisticated statistical methods to compensate for under-sampling: effectively, one makes weighted estimates of under-sampled populations based on known demographic distributions. This generally increases the error term, so one increases the sample size to bring error terms back to the conventional ±3%. I don't know what methods Pew or 538 use; I don't even know if their methods are held in secret to protect the companies' interests. Oct 31 '20 at 17:28
  • I know about corrections for demographic weighting. Since this is a cognitive bias, I'm wondering if it would persist even within demographic groups. For example: Are conservatives within any given demographic group of (age, gender and SES), less likely to answer surveys. Oct 31 '20 at 17:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .