All of Nate Silver's State-Level Polling Predictions Proved True 576
kkleiner writes "For the last few months, the political pundit class has been at war with NYT/FiveThirtyEight blogger Nate Silver. Joe Scarborough of MSNBC called him a "joke," while an op-ed in the LA Times accused him of running a "numbers racket." But last night, Silver triumphed: every one of his state-level presidential predictions proved true. "
Math (Score:5, Interesting)
http://www.xkcd.org/1131/
Not how statistics works (Score:3, Interesting)
This actually shows that Silver is poorly calibrated. if he were accurately calibrated, 80% of his 80%-confidence predictions would come true, 50% of his 50%-confidence predictions would come true, etc. But 100% of his >50%-confidence predictions came true. In the future, he should be more sure of his predictions.
Re:Good for him (Score:5, Interesting)
Another possibility is that the Founding Fathers understood just what kind of dumbshits most of "the people" are, and built in some safeguards to allow civilization to survive the inevitable popular vote to shut it down.
Why Nate? (Score:5, Interesting)
Why does Nate get so much attention, when other sites like electionprojection.com and electoral-vote.com do a similar service, are open on their methods and have had almost perfect results for the past two elections. This past election, those two sites only missed on Florida and that one was truly too close to be 90% confident on one way or another.
Re:Good for him (Score:2, Interesting)
Another possibility is that the Founding Fathers understood just what kind of dumbshits most of "the people" are, and built in some safeguards to allow civilization to survive the inevitable popular vote to shut it down.
Not only that, but the restrictions on who could vote (basically white landowners) wasn't there because of some inherent prejudice that suggested women, free blacks or other demographics were lesser. The restrictions where there because white landowners had a very high chance of having a solid education. It morphed over time (both in society and in the "history" books) into something more racist / misogynistic. The only thing worse than pure "majority rules" is "uneducated majority rules".
Re:Math (Score:5, Interesting)
This topic isn't nearly as wild as the election the year I was born. From USA Today's In '52, huge computer called Univac changed election night: [usatoday.com]
That doesn't take away from Silver's math, though, considering that the polls all had Obama and Romney neck and neck and Obama won by a huge margin. It seems Woodbury did a far better job with an incredibly primitive computer than the modern polsters' statisticians did with today's high tech machines.
Big data PROVED 2000 and 2004 elections stolen too (Score:2, Interesting)
The opinion polls entering election day and exit polls proved the Karl Rove and the GOP stole both the 2000 and 2004 presidential elections for George W Bush.
Re:Math (Score:4, Interesting)
Not this.
No one was pretending that polls are some kind of random guess by someone. The thing is that polls and elections are different circumstances. When you vote, you need to travel and you can do it in secret. It also counts so maybe you change sides when push comes to shove. There's also fraud, which isn't exactly polled for either.
The ultimate question isn't whether the polls were "wrong", but whether they could accurately predict the outcome of an election, which is not the same as a poll. In a computer context, this is like using synthetic benchmarks to predict real-world performance. Sometimes with good data and a good model you can nail it. Sometimes you overlook something and are fairly off.
In reality, I'd say it's damn remarkable that the outcome was so close to the predicted value. From the perspective of fraud _alone_ this is a striking result: either the statisticians can predict fraud quite well or it's not as much of an issue as expected. (Or, maybe the shadow organization controlling the outcome of elections got lazy and decided to just follow the predictions this time around :p.)
Re:Math (Score:5, Interesting)
"That doesn't take away from Silver's math, though, considering that the polls all had Obama and Romney neck and neck and Obama won by a huge margin."
That's the thing though, Silver's math was extrapolated directly from the polls themselves. The polls weren't really close; that was just an illusion created by bad interpretation of the data. Media pundits cherry picked the closest polls, fudged margins of error, and made differences of 1% sound very small when in reality they represented hundreds of thousands of votes.
That's the whole point. The election wasn't ever really close at all. The evidence was there, most people just interpreted it badly (or outright dishonestly).
Re:Good for him (Score:4, Interesting)
Re:Good for him (Score:4, Interesting)
The electoral system causes presidential candidates to entirely ignore EIGHTY PERCENT of the population. Does that sound right to you? I think it is perfectly reasonable for a candidate to spend a lot of time in California, Texas, New York, and Florida. Because that's where most of the constituents are.
It doesn't ignore eighty percent of the population. It does precisely what it was intended to do -- ignore the entire population.
Electing the president, in the US, is done by the electors associated with the states, not by the people. That's just like how a Prime Minister is elected by the MPs in the UK, or most other elected officials. Why? Because the population is a bunch of morons who aren't qualified to pick a leader.
The fact that you can vote for anything other than positions associated with your state is a tradition the states created, but was *not* the intent of the founders of the US. Go read the 12th amendment -- you may be surprised what it contains, if you haven't. There is absolutely nothing about the people voting for the President, and there was no intention for that to be the case.
The US was founded under the belief that voters needed to be educated, and best understood their local issues, and the people elected in the states had the job of understanding the broader issues. But no one seems to actually learn about the structure of the US government or the reasons why it was carefully structured as it was anymore.
The thing that is messed up in the US isn't that a handful of voters in swing states are picking the president, its that the teeming masses of mouth breathers no matter where they live have any say in it. We've cut out the layers of indirection that were put in place explicitly to keep a more stable central government.
Re:Pundits aren't there to provide accurate data. (Score:4, Interesting)
I'm sorry, but this was all just smoke and mirrors. Romney never had the kind of support he'd need to win. I never saw a poll that firmly put him in the lead in any battleground state, or on the national level. The people who were saying he had a chance were the people with a vested interest in saying that. The two political parties were saying it in order to get their voters to the polls, the news agencies were saying it to earn viewers. No one who was being totally honest about it wold have thought Romney had a chance. His base hated him, and the opposition hated him even more.
Re:Math (Score:4, Interesting)
The Tea Party is an aberration, but the Tea Party is an attempt to infiltrate and hijack the Republican Party- all Tea Party candidates are registered Republican. Should the Tea Party identify itself as its own unique party at this point then it would find itself in the same position as the Libertarians, with no national apparatus to help rearrange funding and poor name recognition. Granted, it would start with a position of relative strength given the number of Republican/Tea Party members that are in office, but without national support as part of the Republican party they'd probably lose elections fairly quickly. If they manage to more thoroughly take over the party, though, they might either be able to strip off the apparatus for a true separate Tea Party, or just make the Republican Party itself the defacto Tea Party in whole.
Re:Math (Score:5, Interesting)
Pundits were creating the illusion of close races to drive up viewing.
Of they are simply delusional - to the very end. Example: Live Fox News coverage exchange between Meghan Kelly and Karl Rove just after Fox predicted an Ohio win for President Obama and, thus, the election. Rove said they were flat-out wrong and Kelly said:
Is this just math you do as a Republican to make you feel better, or is it real?
Karl assured her that his "math" was real. Kelly then trotted off to talk with the statisticians who explained their math and stood by their results with "99.95% certainty."
I really hate to say this, but "Yay Meghan" and, except for the talking-head pundits they had on, Fox actually did a rather professional job of covering the election (I flipped through all the major channels), though it was probably because they were expecting a huge Romney win... (especially considering how quickly they signed off after Obama's acceptance speech)
Re:Math (Score:5, Interesting)
That doesn't take away from Silver's math, though, considering that the polls all had Obama and Romney neck and neck and Obama won by a huge margin.
Actually, there wasn't all *that* much math... and in fact (given most of his raw data was just the polls) - the polls that mattered did *not* have them neck and neck. His key insight was just to use polling information by state, find the bias in some polls (like Rasmussen, which had Romney by 2%, hah!) and then weigh and average those polls to get predicted electoral votes. I bet it's simple enough computations Univac could chug through it in a reasonable time :)
It's basically a given now that future presidential race predictions will be based on those same ideas... in fact, the Princeton predictions use a similar model and came up with pretty much the same results:
ELECTORAL PREDICTION (mode): Barack Obama 303 EV, Mitt Romney 235 EV. The mode is the single most frequent value on the EV histogram. It corresponds to the map below, and has a 22% chance of being exactly correct. The next-most-likely outcome is Obama 332, Romney 206 EV.
[and unlike Silver, their blog [princeton.edu] goes into all of the gory details of their model, which is pretty cool...]
Re:Math (Score:4, Interesting)
That doesn't take away from Silver's math, though, considering that the polls all had Obama and Romney neck and neck and Obama won by a huge margin.
Actually, it wasn't "all the polls" that said that, just some of the polls. His math actually took all the polls into consideration and their vectors, which is the point. Additionally, while Obama won by a large Electoral College margin, the Popular Vote was rather close - something the Republicans will try to remind us all of in the months to come as they try to find some grip on reality.
Re:Good for him (Score:4, Interesting)
I'm saying your suggestion has the exact same flaws as the current system...
Current system: California gives all 55 EVs to Obama despite nearly 40% of the state/districts wanting Romney.
District system: California give rougly 35 EVs to Obama and 20 EVs to Romney, which is roughly in line with how the electorate voted.
You use this phrase 'exact same', but I'm not sure you understand what it means.
...and won't make people feel any less disenfranchised.
I think those people that voted for Romney who now see their vote having some effect on the outcome of the election would strongly disagree with you.