What is the typical error in large vote counts?

Question

Well the EU referendum happened in the UK and there was some talk about what happens if the result is a tie, or if it is very close. Apparently, there would not be a recount unless there was a specific problem with the counting in one area. I am not interested in the legal questions of what happens if the result is close. The person who has the largest number of declared votes wins. That's how it works.

What I am interested in whether anyone has estimated the error on the final results? When people count up all the votes, they make accidental mistakes. If you counted all the votes ten times you would likely get ten different numbers which is why people do recounts to check that the result is correct.

Has anyone ever looked at data from recounts in previous elections, either in the UK or elsewhere, to estimate the sort of standard deviation that we can expect. If you do two recounts and the results differ by a hundred votes in ten thousand, is this statistically significant or is it within the expected error from human counting?

I hope the question is clear now. Please ask if it is not.

Answer 1

Margin of Error

There is no error margin for vote counting.

Margin of error is a measurement of uncertainty that results from sampling. When votes are counted, they don't pull a random of sample of ballots to count; they count all of them. Therefore, there is no margin of error.

Measurement Error and Research

However, there may be measurement error. This is the error introduced by measuring (counting) votes. I searched through some political science journals (through JSTOR) and didn't find anyone who has done the research you want.

This is likely because there is no good example to study. In order to do this research correctly, you would need a single election with a large number of recounts. This doesn't happen, ever. It's unlikely to have a single recount, let alone enough to generate some useful inferences.

Why not look at a large number of elections each with one recount? Because with only two observations you won't be able to tell anything useful. It's entirely possible that with only two observations, both of them are unreasonably high or low. In the best circumstance, your conclusion would only be a range of possible differences between the two vote counts ("95% of recounts resulted in a difference of between 1%-3%", for example). This doesn't tell you anything about what the true value of any of the elections are.

Answer 2

I think that given the recounts that have occurred historically, such an analysis is possible. But I don't know if it's actually been done.

Here's a very quick and simplistic analysis using two historical examples.

In the 2004 Washington governor election, there was a machine recount and a manual recount. Each recount changed each of the leading candidates' vote shares by about 0.003%.

Gregoire: 48.8666% → 48.8702% (+0.0036%) → 48.8730% (+0.0028%). Rossi: 48.8759% → 48.8717% (-0.0042%) → 48.8685% (-0.0032%).

In the 2008 Minnesota senator election, there was one recount. The average (absolute) change was about 0.007%.

Coleman: 41.988% → 41.984% (-0.004%). Franken: 41.981% → 41.991% (+0.01%).

So based on these very limited data, we might say that the measurement error is typically between 1 in 100,000 and 1 in 10,000.

There are plenty more data from recounts throughout history. Also, we can look at county-level data. Hopefully some academic will take on this task and do a more proper analysis.

P.S. Another very useful example would be Florida 2000. There was subsequently an academic project to get a better count of the vote (Florida Ballots Project), so you can compare the counts from Nov and Dec of 2000 to the count from the Project.

Answer 3

Something like this was asked in a statistics related group. First, in an election, "statistically significant" doesn't matter. A majority is a majority. A majority of one vote is a majority.

A different question: In the UK, about 51.9 percent of voters voted to leave the EU. If you picked about 30,000 voters absolutely randomly, the chance that less than 50% of these 30,000 would vote to leave would be less in a billion.