Slashdot is powered by your submissions, so send in your scoop

 



Forgot your password?
typodupeerror
Math Politics

Daily Kos Pollster Made Up Numbers 546

Posted by kdawson
from the transparency-wins dept.
jamie found a story up on Daily Kos revealing that the polling firm they had contracted with for 18 months, Research 2000 or R2K, apparently made up or at least manually tweaked its polling results. The blog published a preliminary report by a team of statistics gurus (Mark Grebner, Michael Weissman, and Jonathan Weissman), and it is an exemplar of clarity and concision. The team reports, "We do not know exactly how the weekly R2K results were created, but we are confident they could not accurately describe random polls." Daily Kos will be filing a lawsuit against its former pollster. "For the past year and a half, Daily Kos has been featuring weekly poll results from the Research 2000 (R2K) organization. These polls were often praised for their 'transparency,' since they included detailed cross-tabs on sub-populations and a clear description of the random dialing technique. However, on June 6, 2010, FiveThirtyEight.com rated R2K as among the least accurate pollsters in predicting election results. Daily Kos then terminated the relationship. One of us (MG) wondered if odd patterns he had noticed in R2K's reports might be connected with R2K's mediocre track record, prompting our investigation of whether the reports could represent proper random polling. ... This posting is a careful initial report of our findings, not intended to be a full formal analysis but rather to alert people not to rely on R2K's results."
This discussion has been archived. No new comments can be posted.

Daily Kos Pollster Made Up Numbers

Comments Filter:
  • To be fair... (Score:3, Informative)

    by Anonymous Coward on Tuesday June 29, 2010 @02:20PM (#32735110)
    Nobody expects the Daily Kos to be accurate.

    It would be like trusting Ann Coulter, Rush Limbaugh, Sean Hannity, or anything ever aired on "Air America" before it went bankrupt.
    • Re: (Score:3, Funny)

      by Anonymous Coward
      ..."Air America" before it went bankrupt.

      Financially or intellectually bankrupt?
    • Re:To be fair... (Score:4, Interesting)

      by RingDev (879105) on Tuesday June 29, 2010 @02:33PM (#32735330) Homepage Journal

      [quote]or anything ever aired on "Air America" before it went bankrupt.[/quote]

      I actually liked Rachel Meadow on Air America. Every night she would give the daily death tolls from Iraq and Afghanistan. Something that no other news/talking head program that I have been able to find on my radio dial does. The rest of the line up was pretty 'meh' though.

      -Rick

      • Re: (Score:2, Insightful)

        by ccarson (562931)
        I wonder if she still does that at the beginning of her program? It seemed that when Bush was in office, the left screamed bloody murder when it came to the war(s). Now that their guy is in office, you can hear chirping from crickets.
        • Re: (Score:3, Insightful)

          by Tumbleweed (3706) *

          I wonder if she still does that at the beginning of her program? It seemed that when Bush was in office, the left screamed bloody murder when it came to the war(s). Now that their guy is in office, you can hear chirping from crickets.

          She doesn't do that on her tv show on MSNBC, but she goes after Obama all the time on lots of other stuff, as does Keither Olbermann, Jon Stewart and Stephen Colbert. To think that 'the left' doesn't go after Obama for anything is to completely ignore some of the biggest names

        • Re:To be fair... (Score:5, Insightful)

          by skids (119237) on Tuesday June 29, 2010 @03:53PM (#32736560) Homepage

          Olbermann continued to close his show with the number of days since "Mission Accomplished" right up until a month or so ago, when he switched to the number of days since the deepwater horizon leak started. Much of the left does not pull punches against Obama for taking his time extracting us from these debacles.

          The far left idealists can get quite heated against Obama. Me, all I have to do is imagine how McCain would have responded, then after I've wiped off all the cold sweat and stopped gritting my teeth, I have no regrets about 08.

    • Re:To be fair... (Score:4, Insightful)

      by Anonymous Coward on Tuesday June 29, 2010 @02:37PM (#32735396)

      Perhaps you misread the summary? The Daily Kos is not at fault here.

    • Re: (Score:3, Informative)

      Actually, I do. And now They're suing the pants off of R2K. [fivethirtyeight.com]

      If this was the National Review Online, or Free Republic, or what have you, there would be a huge push to cover this up and blame the "liberal media"(whatever the hell THAT is) for any accusations that they did something wrong.

      • Re: (Score:2, Insightful)

        by osu-neko (2604)

        Actually, I do. And now They're suing the pants off of R2K. [fivethirtyeight.com]

        If this was the National Review Online, or Free Republic, or what have you, there would be a huge push to cover this up and blame the "liberal media"(whatever the hell THAT is) for any accusations that they did something wrong.

        I doubt they would have questioned the results to begin with, much less investigated...

    • Re:To be fair... (Score:5, Insightful)

      by Angst Badger (8636) on Tuesday June 29, 2010 @03:28PM (#32736124)

      Nobody expects the Daily Kos to be accurate. It would be like trusting Ann Coulter, Rush Limbaugh, Sean Hannity, or anything ever aired on "Air America" before it went bankrupt.

      On matters of fact, they're pretty scrupulous, especially when it comes to owning up to their own mistakes, like hiring R2K.

      On matters of opinion and ideology, well, it's a political blog. What exactly is an "accurate" opinion?

    • Re: (Score:3, Funny)

      by CAIMLAS (41445)

      Actually, Rush is pretty accurate. He's just (admittedly) biased. It's the bias that gets you - what he's typically saying is not factually inaccurate.

      If he were publishing statistics, on the other hand, the bias would be a problem. Thus the Daily Kos credibility comes into question...

  • by Pojut (1027544) on Tuesday June 29, 2010 @02:21PM (#32735138) Homepage

    Those who aren't used to phrases used with "political" centric organizations might mistake the title as saying someone who is on Daily Kos' payroll flubbed the numbers, rather than a company working on contract with them.

    • Re: (Score:3, Informative)

      by Ocker3 (1232550)
      even simply adding an apostrophe to make it "Daily Kos' pollster made up numbers" would be more informative
      • Re: (Score:3, Informative)

        by blueg3 (192743)

        Unless Kos is plural, since it's not the name of an ancient person, they'll need to add an apostrophe and an s. "Daily Kos's Pollster Made Up Numbers".

        • Re: (Score:3, Informative)

          by Volante3192 (953645)

          It's technically correct (the best kind of correct) that you can show possession with simply an apostrophe for singular nouns ending in an 'ess' sound.

          The trick is you have to be consistent about it. (You can't start with "Daily Kos' pollster" and later use "Daily Kos's editor".)

          It's more of a guideline than a rule to use the succeeding 's.'

          However, I will say that leaving the 's' off would likely be a depreciated style if this was a standards documentation.

  • Polling (Score:3, Informative)

    by Mongoose Disciple (722373) on Tuesday June 29, 2010 @02:26PM (#32735220)

    For me, the surprising part of this isn't so much that R2K made up poll results, but that the results actually were noticably less accurate than traditional polling, which I like to think of as representing a broad cross-section of people who still have landlines with no caller ID for some reason (or are desperate enough to talk to another human being that they'll answer their landline anyway).

  • Give them credit. (Score:4, Interesting)

    by Anonymous Coward on Tuesday June 29, 2010 @02:27PM (#32735238)

    Unlike the many Republican outfits which used partly- or wholly-fabricated polls by Strategic Vision, or the many media outlets which continue to use the horribly flawed Rasmussen polls to create eye-catching headlines, Kos immediately dumped the pollster, did an investigation, owned up to the errors publicly, and is now pursuing legal recourse.

    This is exactly how you would expect an honest media organization (if one with a considerable political agenda) to behave. Too bad the mainstream media and those on the other side of the aisle don't seem to want to do the same.

  • Gee (Score:3, Funny)

    by BCW2 (168187) on Tuesday June 29, 2010 @02:31PM (#32735290) Journal
    You'd think they would spend Soros money more carefully.
  • by Zephyr14z (907494) on Tuesday June 29, 2010 @02:32PM (#32735314)
    I used to work a different major polling company, and I can assure you R2K is not alone in just making up numbers. Easily 80% of surveys that went through my region were completely falsified, and the remaining 20% rarely matched the demographic they were supposed to be answering for. Survey administrators have quotas, and then get paid extra for additional surveys past that, but there is basically nothing done to verify any of the surveys turned in, and everybody in the company knows it. Don't always trust what you read, especially not statistics.
  • by emagery (914122) on Tuesday June 29, 2010 @02:34PM (#32735348)
    For headline skimmers, this post would produced a completely inaccurate sense of what the article was all about... at length, the D.KOS are the ones who found out about this and are doing something something about it. That's good... but if you just read the headline, you'd come away thinking that D.KOS were the culprits instead.
    • by Scareduck (177470)
      I don't get that. "Daily Kos Pollster" is the pollster used by the website "Daily Kos". If you read the story, Kos was defrauded by the pollster.
      • I don't get that. "Daily Kos Pollster" is the pollster used by the website "Daily Kos". If you read the story, Kos was defrauded by the pollster.

        But it doesn't emphasise the pollster was a third party. Replace 'Daily Kos' with 'Fox News' or 'MSNBC' and see how it can be (mis)interpreted.

        If you need the DK reference, then 'Pollster for Daily Kos...' is better as it implies more a working relationship rather than part of the organization.

        Or they could have just said 'Research 2000 Makes Up Poll Numbers' Don

    • Re: (Score:3, Interesting)

      by tomhath (637240)
      Kos' sponsors paid R2K for a year and a half because they got the results they wanted to hear, even though everybody knew the results were complete BS. They finally got called out on it so they go into damage control mode....That's good?
  • of course this will turn into a "bash the left" and a "bash the right" thread. when ideology isn't the point. polling is the idiocy in question

    and the guy who manipulated the numbers is clearly an amateur. the way you do proper poll manipulation is LOAD THE QUESTION. you poll people with a question with the proper turn of phrase to lead them towards the answer you want. then, when you present the answers to the poll, you also cage the results in such a way to lead the audience in the way you want them to interpret the results

    polling is fucking joke. all results from the left, or the right, is complete bullshit, and a waste of your time

    • by mea37 (1201159) on Tuesday June 29, 2010 @02:54PM (#32735640)

      You're assuming the motive was to manipulate the outcome.

      Did it not occur to you that maybe the motive was to provide any outcome that would look real enough to get paid, while not doing as much work?

    • by Abcd1234 (188840) on Tuesday June 29, 2010 @02:59PM (#32735710) Homepage

      polling is fucking joke. all results from the left, or the right, is complete bullshit, and a waste of your time

      Or, to put it another way, it's absolutely impossible to know what the people want without asking every single one of them.

      Genius.

      No, wait, sorry, I meant "bullshit". Polling is a tool, and an extremely important one. Can it be done very poorly? Yes, of course, But that needn't necessarily be true. And it's the only option for understanding a population when there's millions and millions of individuals.

    • by Myopic (18616)

      Depends on what you mean by "joke". Using polling data, Nate Silver predicted the 2008 election within less than a half a percent. That's pretty accurate, but it could still be a joke for some definitions of joke.

    • Re: (Score:3, Informative)

      by Entropius (188861)

      Daily Kos wasn't trying to manipulate anything -- notice that they fired R2K once fivethirtyeight's statistics showed them to be least accurate at predicting election results, long before there was any evidence of fraud?

  • Remember: (Score:3, Informative)

    by Locke2005 (849178) on Tuesday June 29, 2010 @02:50PM (#32735578)
    If you place a statistician's head in ice and his feet in boiling water, then on the average he is quite comfortable!
  • by coaxial (28297) on Tuesday June 29, 2010 @03:03PM (#32735758) Homepage

    In Neal Stephenson's Cryptonomicon [wikipedia.org], there's a scene early in the book where the Allies are assembling the personnel for Station X (aka Bletchely Park [bletchleypark.org.uk]). Statistician, turned Nazi codebreaker Lawrence Waterhouse, points out that his Nazi counterpart Rudy von Hacklheber, would notice something was amiss with the Allied personnel changes based the statistics of people being transfered to Bletchely Park, and then quickly deduce that the Allies are attempting to break the Enigma code. To camouflage the transfers, Waterhouse suggests creating ficticious personnel and have some of them transfered to Bletchely Park as well. However the military can't just make any random fake person, the fictious people must be statisitically drawn from a distribution that when added to distribution of real Bletchely Park personnel, the combined distribution is statistically insignificant [wikipedia.org] (i.e. fail to reject the null hypothesis) than any other large military base.

    If Research 2000 did what is suggested, they failed to taint the polls with the right kind of fake data, just like what the novel warned about.

    • Re: (Score:3, Insightful)

      by hibiki_r (649814)

      When you are attempting to do as little work as possible and still get the million dollars a year kos spends on polls, mocking with the data in such a way that nothing amiss can be detected is rather counter productive: You might as well do the polls right anyway.

HELP!!!! I'm being held prisoner in /usr/games/lib!

Working...