There isn’t any evidence to support President Trump’s assertion that three to five million illegal votes were cast in the 2016 presidential election. But there is one study that has been interpreted to suggest it is at least possible. It found that between 32,000 and 2.8 million noncitizen voters might have fraudulently cast ballots in the 2008 presidential election. The study, based on a survey of 38,000 people after that election, has been under fire since it was published in 2014. Now even its authors concede that it probably overstated the amount of noncitizen voting. “The high-end estimates are likely incorrect,” Jesse Richman, one of the co-authors of the study and a political science professor at Old Dominion University, said in an email exchange on Wednesday. In a post online, he also said that the findings do not support Mr. Trump’s contention that millions cast ballots illegally. Mr. Richman still maintains that some small percentage of noncitizens vote in American elections. But the debate over this study has moved on. It’s no longer about whether millions of illegal votes were cast, but whether there’s any evidence for noncitizen voting at all. The study’s bold claims fell apart because of something called response error: the possibility that people taking a survey don’t answer a question correctly — in this case, a question about being American citizens.
There is always a tiny amount of response error in surveys. Respondents might not understand the question. Or they might understand it, but mark the wrong answer by mistake, if the survey is self-administered. An interviewer, if there is one, could accidentally record the wrong answer. Such errors usually aren’t a problem large enough to change the results of a survey.
But both the survey and the question posed by researchers were unusual. The survey — the Cooperative Congressional Election Study — was huge, with 38,000 respondents in 2008. And the group in question — noncitizens — was very small, just 339 of those respondents.
The problem is that even a tiny amount of response error among the 38,000 respondents could have been enough to contaminate the results of the tiny group of noncitizens.
Imagine, for instance, that 99.9 percent of people respond to the survey’s citizenship question correctly. In such a big survey, even that high success rate would still imply that there were 38 respondents who answered incorrectly — enough to make up a big chunk of the tiny pool of 339 noncitizen respondents. If those 38 misreported noncitizens had indeed voted, then suddenly it would look as if 10 percent of noncitizens voted.