All of Nate Silver's State-Level Polling Predictions Proved True 576
kkleiner writes "For the last few months, the political pundit class has been at war with NYT/FiveThirtyEight blogger Nate Silver. Joe Scarborough of MSNBC called him a "joke," while an op-ed in the LA Times accused him of running a "numbers racket." But last night, Silver triumphed: every one of his state-level presidential predictions proved true. "
But when? (Score:5, Insightful)
As Nate Silver pointed out, many times, over the course of the campaign, predicting what will happen one day before the election is easy. Very easy. Most everyone gets that right.
Predicting what will happen in June is hard. And much more interesting.
Seriously... (Score:5, Insightful)
Just to be clear, these are statistics. (Score:5, Insightful)
You can't really prove a probability wrong (unless it's 0% or 100%). While his most likely outcomes played out, it doesn't mean that he would have been wrong if a few of them hadn't.
Comment removed (Score:5, Insightful)
Re:Good for him (Score:4, Insightful)
Too bad the electoral system in the USA is a joke and doesn't represent the vote of the people.
So you wanted Al Gore to be president in 2000?
Re:Good for him (Score:5, Insightful)
I wish people would stop blaming the electoral college. The system is fine. It is the method in which the individual states assign the votes that is the problem. Florida in 2000 wouldn't have been such a big deal if they had distributed the electoral votes by district, rather than winner take all.
Win California by one vote? You get all 55 electoral votes! How stupid is that?
Re:But when? (Score:5, Insightful)
And yet, even the very day of the election, there were still large numbers of pundits predicting a landslide victory for Romney. Guess the predictions aren't that easy, or perhaps it's just easier to ignore the numbers and resort to wishful thinking.
Re:Math (Score:5, Insightful)
This. People need to learn that statistics and polling are sciences. Like all sciences they are inexact, with a margin for error; but the chances of the poll averages being wrong in this case were incredibly small.
Or This. Pundits and Statisticians are about as far apart as Republicans and Climate Scientists.
Re:Good for him (Score:2, Insightful)
It's as stupid as going with votes by district.
Win each district by one vote? You get all the votes, and hence all the states and the election! How stupid is that?
Pundits aren't there to provide accurate data. (Score:5, Insightful)
Pundits are there to draw people to a news organization, not provide accurate information. How many people would tune into the election if they said "Obama's got this one in the bag" (which we've known ever since Romney was nominated). Of course they will say you don't know who's going to win! Otherwise no one will watch their show.
Re:Not how statistics works (Score:4, Insightful)
There are still warning signs. For example, he has issued predictions with three digits of precision. That's an obvious sign that something isn't right.
Re:Good for him (Score:4, Insightful)
There's only one President, and the guy with the most votes cast for him is there.
How is this NOT representative?
Re:But when? (Score:5, Insightful)
Why do you think the early media reports were landslide for Romney? Why do you think it stayed so tense? Why do you think states were called before they were reporting, or that they were called for Romney at 74% reporting with 60% going to Obama? 97% chance of Obama victory. Obama's gonna win, Obama's winning, cool. Let's put in the Harry Potter DVDs and make some popcorn.
Don't watch Harry Potter, you faggots! Watch MSNBC and CSPAN so we can get ratings! Hey! HEY! HEY, CHECK IT PEOPLE, ROMNEY IS WINNING!
Re:Good for him (Score:5, Insightful)
Re:Good for him (Score:5, Insightful)
David Brooks gave an interesting response, he said that the electoral system forces the candidates to make an effort to play somewhat to the middle. Without the electoral system, Barack Obama would have campaigned heavily in California to get the liberal count up there. Mitt Romney would have campaigned heavily in Texas to instigate the conservative vote. The result was that they needed to go to places that weren't exactly on their side and try to convince them. They were forced to answer questions that both sides wanted to hear an answer to, rather than just their base. (I dare say most people haven't been pushing for real answers, but that's another issue altogether)
Re:Math (Score:3, Insightful)
; but the chances of the poll averages being wrong in this case were incredibly small.
I'm not sure that's exactly knowable. Sure, the numbers are way better than contradictory pundit guts, but for instance, we had no way of knowing if a "Bradley Effect" would have been in play. Response rates for polling firms consistently came in below 10%. Polling is getting harder and harder in an age where fewer people have landlines and polling cell phones is restricted. As of now, state polls are good guides. They will be right up until they aren't, and then the science will change.
I'm not saying that the probability of systematic error is large, just unknowable. It was a perfectly reasonable and scientific position for a Republican to say "Romney's chances are equal to the probability of error in the polls, and I hope that probability is large."
Re:But when? (Score:5, Insightful)
Re:Why Nate? (Score:5, Insightful)
Re:Not how statistics works (Score:4, Insightful)
Congratulations, this is the stupidest thing I have read today.
The confidence rate of a prediction doesn't work like that. It's the probability with which the null hypothesis can be rejected given the data, basically, suppose that the relationship you are trying to prove isn't there, how likely it is that the data were generated by a statistical fluke? And much like any other statistics, a bunch of predictions with a 50% confidence interval doesn't mean that half of them must come right, especially in a single sample - all it means is that it's as likely for the theory to be true as for it to be false.
Re:Math (Score:5, Insightful)
We can know the odds to a high precision. Just like I can know the odds of the flip of a weighted coin. That doesn't mean I can't be wrong when I guess my next coin toss, it just means I have a fair idea how likely I am to be wrong.
Re:Math (Score:5, Insightful)
They could care less if it adversely affected a vote or increased voter turnout.
The fact is they manipulate the information for their own personal gain. This time was it not only Fox News and MSNBC, but CNN.
This opens up an avenue for truth, which sheds light on what they are doing. Media outlets are calling him that because they are upset he is telling the actual truth and not making it cloudy to increase revenue.
Colbert Report made fun of this when he had him on the night before the election.
Re:Good for him (Score:5, Insightful)
The result of the vote is that he won. The result of the electoral vote is that he won.
This is pretty much the textbook definition of whining about something that doesn't matter in the slightest.
Re:Math (Score:5, Insightful)
Indeed. There was an illusion of a close race to sell advertising. People love drama, and having a contest where the media reported "Obama's got it, Romney's cause is hopeless" would not have had the sexy urgency necessary to cash in on.
We are seeing the OJ simpson freeway chase kind of reporting being applied to elections.
Re:Not how statistics works (Score:5, Insightful)
This actually shows that Silver is poorly calibrated. if he were accurately calibrated, 80% of his 80%-confidence predictions would come true, 50% of his 50%-confidence predictions would come true, etc. But 100% of his >50%-confidence predictions came true. In the future, he should be more sure of his predictions.
Not necessarily. Most of the uncertainty in his predictions was due to the conditional probability of systematic bias in likely voter models. For example, Gallup was showing much better results for Romney and the Rac... err... Republicans across the board, which was probably due to how they screened people who responded.
Systematic error shows up as a conditional probability, so you are lumping together completely disjoint realities into your final result. In terms of discrete conditional probabilites, imagine that based on historical data you have three equally likley possible conditions: 5% Democratic bias, no bias, and 5% Republican bias. You run your simulations with each of these three biases, and you get a result that says senators D0, D1, D2, D3, D4, D5, D6, D7, D8 and D9 are all 80% likely to win. But 99% of that 20% chance they'll lose comes from the condition where there's a 5% Republican bias.
But remember: those three conditions describe completely disjoint realities. They are not sampling error, but statements of ignorance about the actual state of the world.
Now the world really is just one way (it may be ambiguous relative to some human categorization, but then that ambiguity is just part of the one unique way the world realy is.) So only one of the three conditions are true. If it happens that the no-bias case is the way the world really is, then 100% of those 80% chances will come true.
That said, in future elections Bayesian predictions of the kind Silver and everyone else in this space are making will lower the conditional probabilities of bias, because this election demonstrated good low-bias results, but so long as the ultimate uncertainty is dominated by the systematic error, Bayesian predictors will tend to appear either uncannily accurate or dismayingly inaccurate.
However, averaged over many, many election cycles (18 or more) you would expect to get statisics such that 80% of the 80% calls are correct, and so on. But within individual elections that use fixed likely-voter models that won't be the case.
Conditional probabilities are one of the most difficult things for humans to understand (the Monty Hall problem is a classic case where all the confusion comes from treating a conditional probability as if it was a total probability) so it's worth practicing the art of thinking carefully about these things, and the odds are still good I've said at least one confusing or incorrect thing in the above.
Re:Math (Score:5, Insightful)
Part of the problem was an obsession with national level polling. Silver was analyzing the races per state, which is the only legitimate way to analyze it in an electoral college system. National polling is at best only an extremely crude indicator, and to my mind, in most modern presidential elections is likely useless.
Re:Good for him (Score:5, Insightful)
The Electoral College system was never meant to represent "the will of the people". The House of Representatives is supposed to represent the will of the people. The presidency and the Senate had entirely different purposes and mandates.
Re:Good for him (Score:5, Insightful)
This could be framed another way; that is, "swing states" are the only ones the candidates care about. The rest of us are taken for granted.
Re:Good for him (Score:4, Insightful)
Yes! Given what a disaster Little Bush turned out to be it Al Gore would probably have been a significant improvement.
Re:Math (Score:5, Insightful)
This is the price of allowing 10,000 independent journalistic voices to be consolidated into 2 or 4 mega-media-conglomerates who's infotainment is supposed to pass for a free and responsible forth column. What we now call news is to free journalism, what the grocery weekend throw away is to journalistic press. The current media is selling you ideas and opinions. It is paid for and owned by its corporate sponsors, so they will be the source of your information. Anyone who isn't getting news from outside of the United States (and from diverse sources) would be better served reading their toilet paper, in the end it will perform the same job.
Nobody who reads "REALITY" is by any means surprised by any of this, or the really shocking things our government is doing. In the debates, where were the questions regarding the gutting of the constitution or the fact that the President now has a license to kill? Where were the questions about the government printing trillions of dollars to "FAKE" the existence of an economy? Or even the questions about all our trade partners quietly working out new trade routes that exclude the use of dollars? Boys and girls, the fan and the schist are on the verge of close embrace, and ours news hasn't bothered to mention our emperor is prancing about buck nekid!!! I hope y'all have your emergency supplies well stocked... I suggest dehydrated food myself.
Re:Good for him (Score:5, Insightful)
It's as stupid as going with votes by district.
Win each district by one vote? You get all the votes, and hence all the states and the election! How stupid is that?
This needs to be stated as clearly as possible, because it is one side of the greatest flaw in first-to-the-finish voting.
The more districts where the votes are aggregated, the less of the popular vote you need to win.
The more (plausible) options there are on a vote, the less of the popular vote you need to win.
Take a second and read that again. Now I'll explain
First, a district example.
Let's say you have only 1 district, two candidates, and a million people in this imaginary nation. You get 500,001 votes, you're the new president. It's pretty straightforward, and you clearly have the popular vote, if only barely.
Now, let's say we still have 1 million people and two candidates, but we have 10 districts of 100,000 each. To win a district, you need 50,001 votes. To become president, you need 6 districts. Do the math and you only need 300,006 votes to win. While you may have the popular vote, it's not necessary to win.
Now, the candidate example.
Let's say you have only 1 district, two candidates, and a million people in this imaginary nation. You get 500,001 votes, you're the new president. It's pretty straightforward, and you clearly have the popular vote, if only barely. (Yes, it's exactly the same as above.)
Now, let's ay we still have 1 district and one million people, but we have 10 candidates. Given an unrealistically tight race, you could win with only 100,001 votes. Practically, you'd need more than that. How much more? Well, that depends on how close the candidates are in popularity.
So there you have it. If you think popular vote is the most reasonable way to choose the president (which is NOT the model the US uses), you want fewer districts and candidates, or you want to stop using first-to-the-finish voting.
Also, if you combine the two elements above, many districts and many candidates, the percentage of the popular vote required to become president becomes even lower. For a handy real-world example, see Canada.
For the conspiracy theory lovers out there, a nation in the state listed above with a lax immigration policy is a ripe target for a peaceful invasion. Immigrate enough people to become a third of the population in a third of the districts, wait until they have voting rights, have them vote for who they want to be in charge, and the government is yours. This may be easier if you make a new party for this purpose as it dilutes the voter power of those who aren't voting in concert to overthrow the current government. That's right, 4 million people, good planning, dedication, and 5 to 10 years, and a nation the size of Canada could have a peaceful revolution. Conspiracists, you may start to spin your tinfoil hats!
Re:Math (Score:5, Insightful)
But his analysis of national polls was correct too. He predicted Obama would win the popular vote by 2.5 points and he won by 2.4 points. Media outlets were rounding poll results to get "50% to 48% with 1% margin of error OMG it's practically tied!!!"
2.4% sounds like a small difference only until you realize it represents about 2.9 million votes. Pundits dumbed down the math and ended up looking dumb.
Congratulations Nate! (Score:4, Insightful)
I applaud Nate's effort to use mathematics to effectively make fools of all the talking head "pundits" in the media. I have followed him since 2008 and am looking forward to many more years of his work. I hope he continues to be successful in the future.
Re:Math (Score:5, Insightful)
Not exactly. What you say is partly true. But there's another dynamic at play here: when predicting election outcomes, there are two sources of uncertainty, not one.
The first is random sampling error, which is what Nate Silver does an excellent job correcting for.
The second is uncertainty in how likely it is that someone will vote. This means if supporters of a candidate acknowledge that their opponent has a seemingly insurmountable lead in the polls, they create a self-fulfilling prophecy. If their candidate seemingly has no chance, then what's the point of voting for him? His supporters will stay home on election day, and he ends up losing even if the polls were wrong and he was actually in the lead.
So when it comes to elections, you basically have two choices. Hold a gun to everyone's head and force them to vote. Or everyone has to pretend their favored candidate could win, even if the polls show he's losing. When people don't do the latter, you get the situation we have in the U.S. - where people who would really prefer the Libertarian candidate end up voting for a Democrat or Republican. Because everyone "knows" the Libertarian candidate could never win. (There are other ways to combat this, e.g. instant run-off voting, but that's a different discussion.)
Re:Good for him (Score:4, Insightful)
That's half a million voters whose votes basically did not count, and THAT is why the electoral college system is a problem.
Nowhere in the constitution or the intentions of the founders of the US is there anything saying the opinion of the voters SHOULD count towards who is President. In fact, they explicitly set it up to ensure they *didn't*.
The president is the CEO of the US, and the states are the board of directors. You don't see employees of corporations voting for their CEO. Why? Because the vast majority of employees aren't qualified to determine who would make a good CEO.
Re:Math (Score:4, Insightful)
But this is almost the way it has to be done when you have a large population with many secular events – that is when the facts on the ground change.
You have a host of polls. Some have more rigor than others. Some are rolling (ask the same person 2 weeks apart.) Some have demographic data. Some are instant, some take a week to gather the data.
And during that time you have real events happening which is changing the game. Take the 1st debate where Rommey beat Obama. How much weight do you give to poll which spans the first debate, but is big and rigorous against a smaller, sloppier instant poll?
One has to make a subjective judgment based on knowledge of math, the strength and weakness of the various polls, experience, and wisdom.
This is an issue for all social scientist who use statistics. It’s better than you intuition but there are limits.
Re:Math (Score:5, Insightful)
That doesn't take away from Silver's math, though, considering that the polls all had Obama and Romney neck and neck and Obama won by a huge margin.
But the polls didn't have them neck and neck, if you looked at the state level and added up the electoral votes. That's what Silver's math was based on. He does have some non-poll information in the mix too, but Princeton Election Consortium got the same results using pure polls.
Re:Math (Score:5, Insightful)
Instant run-off is the pretty much the single-winner preference voting system that does the least to mitigate the problems with first-past-the-post elections that preference voting systems are usually offered to resolve. About the only criterion I can think of under which it is arguably the best is ease of understanding for people whose only prior experience with voting systems are with variations on first-past-the-post like plurality and majority-runoff. (Which, to be fair, isn't completely unimportant.)
Re:Good for him (Score:4, Insightful)
Note that a consequence of this is that district-level gerrymandering, which already produces disproportionate congressional delegations, would then also produce disproportional EVs from state. That is, Ohio, a 51/49 state, would have had something like 14 Repub. EVs and then Dems would have had 4.
Re:Math (Score:5, Insightful)
For example, when the media was shocked by the 2008 crash, I wasn't. I had predicted it 5 years earlier (not necessarily when, but the fact that it would happen).
You didn't predict anything. You made a guess. It took five entire years for reality to coincide with your guess. That's all. When you can make accurate guesses consistently over time, then maybe we can talk about calling them "predictions".
Re:Math (Score:3, Insightful)
People need to learn that statistics and polling are sciences.
There are, unfortunately, a large number of people, many of them on Slashdot, who will never, ever be convinced of this. They have this dimly remembered high-school "science class" idea of what science is, and that idea doesn't include uncertainty. You can often find them trashing large, well-designed studies by claiming that the scientists who published the results of the studies didn't follow The Scientific Method, which in their minds is this checklist which must be followed and ... ta da! Science happens! And if you don't follow (their idea of) the checklist, then they know you can't really be doing science, because they memorized TSM in tenth grade and by God that's the way science works. And all those so-called scientists who aren't following the checklist? Well, they're just a bunch of puffed-up ivory tower eggheads who will say anything to get rich on grant money.
Identification of specific cases of this phenomenon is left as an exercise to the reader.
Re:Math (Score:5, Insightful)
No, the only states that can split their electoral votes do it winner-take-all by district [wikipedia.org], it's not a strict population proportion.
And for good reason: any state that adopted proportional electoral votes would render itself politically irrelevant. If my state has 10 electoral votes, and we award 6 to one guy and 4 to the other, the winner of our state pulls ahead by 2 votes instead of 10. We've just given away 80% of our power to pick the president.
I'm not saying that the winner-take-all system is better -- far from it. But states have a strong disincentive to change.