What does FiveThirtyEight's number mean?
by Dave Winer Thursday, October 20, 2016

I've been following the FiveThirtyEight  polls-plus number through the summer and into the fall. My mood goes up and down with the percentage chance that Hillary Clinton will win. 

I've always wondered what exactly does the number mean?

Look at it this way. If the polls accurately measure people's votes, then there is zero probability that she will lose, because the votes add up to enough electoral votes to win the election. 

So why is there a non-zero chance of her winning? What could happen that would cause her to lose if the chance of winning is accurate at approx 84 percent?

I gave this some thought and listened to Tuesday's edition of the Run-up podcast and came up with this way of expressing it.

The polls do not accurately measure people's votes. 

Here are some ways errors are introduced.

  • Pollsters don't weigh demographics accurately. They make assumptions about what percentage of voters will be white or black; Hispanic or Asian; college-educated or not; rural, urban or suburban; rich, middle-class or poor, etc. Those assumptions are wrong. That's a fact. How wrong? That's part of the uncertainty.
  • People don't always tell the truth. The Bradley Effect. They tell the pollster something that makes them feel good about themselves (I'm voting for Mother Teresa) but in the end they vote for someone bad (Adolf Hitler).
  • People change their minds. Something happens, a new leaked email shows that Hillary has a nasty habit that we think is awful and disqualifies her. Or Trump turns out to be a sexual predator (oops that's already factored in). 
  • Who is likely to vote? A poll respondent who isn't going to vote won't be part of the result, but they might be counted in the polling. You can't simply take them at their word. How should you weigh that?

The statisticians at FiveThirtyEight know about these errors, based on past experience with polling vs actual elections, and have theories about how much they affect the outcome. And that's weighed against how close it is. If 9 of every 10 respondents say they are going to vote for Clinton, no matter how big the errors are, they're going to say she's got a near-lock on the election. Not much chance the errors could be so big as to turn that kind of polling into a losing result. On the other hand if it's just 6 of 10 then the errors matter a lot and the election becomes a tossup, "within the margin of error," a pollster might say.

So when they say it's an 84 percent likely Clinton win, that means the numbers overwhelmingly say she's going to win, enough to give the statistician a lot of confidence in the prediction. How much? 84 percent. Heh.