Become a fan of Slashdot on Facebook

 



Forgot your password?
typodupeerror
×
Government IT Politics

Edward Snowden Kills Team Trump's Conspiracy Theory By Explaining How The FBI Can Quickly Comb Through Email (geekwire.com) 488

FBI director James Comey told Congress Sunday that the further investigation of emails related to Hillary Clinton didn't turn up anything that would cause the bureau to recommend charges against her. The FBI had reviewed over 650,000 emails under nine days. Upon hearing this, GOP presidential candidate Donald Trump and his supported started to question whether the FBI could go through all those emails in such a short period of time. We will never know for sure until the FBI explains its process to us all (which is unlikely to happen), so people turned to Edward Snowden over the weekend for answers. And Mr. Snowden didn't disappoint. From a report on GeekWire: How easy would it be to cull out the duplicate emails? Outspoken journalist Jeff Jarvis posed that question to Snowden in a tweet, and got a quick response: "Drop non-responsive To:/CC:/BCC:, hash both sets, then subtract those that match. Old laptops could do it in minutes-to-hours."
This discussion has been archived. No new comments can be posted.

Edward Snowden Kills Team Trump's Conspiracy Theory By Explaining How The FBI Can Quickly Comb Through Email

Comments Filter:
  • by Anonymous Coward on Monday November 07, 2016 @02:26PM (#53231493)

    ... then we still don't know how plausible it is that they reviewed XXXX number of emails in 11 days, after taking months to review 80,000 emails before.

    • by Kichigai Mentat ( 588759 ) on Monday November 07, 2016 @02:33PM (#53231583) Journal
      Because each email had to go through several Federal agencies to have any retroactively classified information redacted before they could be publicly released.

      In this case we have a trove of emails . Also note what Comey said: he said that this doesn't change their decision with regards to recommending to indict Clinton or not, so that means once they hit this point all they have to do is figure out if Clinton had sent any of the remainder of the emails, which is easily accomplished with a simple search. [newsweek.com]

      Badda bing, easy work.

      • by Anonymous Coward on Monday November 07, 2016 @02:46PM (#53231699)

        Which once again begs the question why Comey broke the FBI guidelines to not insert himself in the middle of the political process, especially so close to the election. His ass should be canned for throwing all that red meat to the Trump campaign 11 days before the election as Clinton was pulling away in the polls. How do you defend yourself against innuendo from the FBI?

        FOX news seemed to be getting daily updates on how Clinton was going to jail immediately after the election from an unnamed FBI source. They would report them in primetime with great fanfare and then retract them Friday morning where the viewership is much smaller with no fanfare.

        The FBI is a mess right now.

        • I doubt Comey has much of a future. Obama won't touch him prior to tomorrow, but come Wednesday, kicking his ass out the door and cleaning up the FBI will need to be a top priority. Congress could help by inserting some prison time into the Hatch Act, so the next time an FBI director decides to play fast and loose with a presidential candidate, he'll think twice.

          • by hajile ( 2457040 ) on Monday November 07, 2016 @03:50PM (#53232343)

            Democrats said he was an outstanding, honest man when he dropped the case (while Republicans decried him as dishonest). When the case came back up, the Democrats and Republicans both completely flipped positions. I don't know if he's playing politics or not, but it seems obvious that everyone's hatred/love is tied to their party rather than the truth.

            In any case, what could he have done differently? He announced the case closed going into election season. If he didn't mention the new evidence at all, then congress would have him for perjury sooner or later. If he released after and Hillary won, everyone would say he killed the investigation so Hillary could win. If he released before and Trump won, he would be accused of bringing up the investigation again so Hillary would lose.

            Given that Hillary looks to win the election, he can claim that his release didn't adversely affect the election. That's about the best outcome he could hope for.

        • Re: (Score:3, Interesting)

          by Anonymous Coward

          Trump nuked his campaign when the "grab them by the pussy" thing hit the news cycle. Clinton, seeing that there was no time for the Republicans to scrounge a new candidate, ran with it and hit Trump hard. So Trump responded with the whole "rigged election" schtick. Hillary responded by having a close aide with a sketchy husband release a bunch more inert stuff, then called in a favor with Comey to publicize it. Now, Trump can't call the election "rigged" without looking like a complete twit. And a week late

        • Comey probably knew about the (apparent) cell in the NY FBI office who have a hard-on about HRC, and knew they were going to leak something, so he decided to get out in front of it with his letter to Congress. I do give him some credit for his subsequent letter yesterday, but it would have been better IMO if he had shut up and let the leaks happen, then come down like a ton of bricks on the leakers.

        • > Which once again begs the question why Comey broke the FBI guidelines to not insert himself in the middle of the political process

          Watch this and you might understand what's going on a bit more:
          https://www.youtube.com/watch?... [youtube.com]

    • by beelsebob ( 529313 ) on Monday November 07, 2016 @02:34PM (#53231587)

      The point is that Trump's supporters have no solid evidence that there was not enough time to review the emails.

      Their extreme view of it is that all 650,000 emails were relevant, and that therefore it should have taken 18 months * 650,000 emails / 80,000 emails = 146.5 months to review them.

      The other extreme of possibilities is that the FBI filtered the emails by "To/From 'Hillary Clinton', date within period of being secretary of state, not a duplicate of any of the already reviewed emails" and the output of the filter was 0 emails.

      The truth is likely to be somewhere between the two, it's also likely to be towards the very low end of the range.

      • by PvtVoid ( 1252388 ) on Monday November 07, 2016 @02:43PM (#53231663)

        Their extreme view of it is that all 650,000 emails were relevant, and that therefore it should have taken 18 months * 650,000 emails / 80,000 emails = 146.5 months to review them.

        Had it actually taken that long, they would have claimed that there was a conspiracy to delay an indictment.

        • Re: (Score:3, Funny)

          by AmiMoJo ( 196126 )

          They seem to prefer these things are reviewed by independent, unbiased reddit users armed with powerful tools like Google and meme generators, oh and of course a pirate copy of Photoshop to put together the infographic spam.

      • Re: (Score:2, Funny)

        by fahrbot-bot ( 874524 )

        The point is that Trump's supporters have no solid evidence that there was not enough time to review the emails.

        You used the words "Trump supporters" and "solid evidence" in the same sentence -- funny. :-)

    • by alvinrod ( 889928 ) on Monday November 07, 2016 @02:46PM (#53231685)
      Presumably part of the difficulty with solving any problem the first time is figuring out a good method to use, implementing it, and testing to make sure it works properly. Once you've solved that problem, it becomes much easier to do it a second time because you know what needs to be done and people have experience doing it so it goes considerably faster. That isn't proof that they were actually capable of going through everything, but if we want to think about it logically, the outcome was always going to be the same.

      First of all, if someone actually had something really damning they would have released it months ago if they had any intention of going public at all. Anything that's immediately and undeniably legally actionable gives you perfect blackmail material that can be used to control the president of the United States. No one in a position to collect that kind of information (blackmail) is going to waste that kind of opportunity. If you want to argue that someone who might have said information wants to release it to cause disarray, there's a more compelling argument that disarray is maximized if you only release the data some time after a Clinton victory.

      It's therefore safe to assume that there's no silver bullet in the new data dump to start with and that it only contains more of the same, which the FBI have already said isn't going to get anyone to indict Clinton, even though they've essentially stated she's been pretty duplicitous about the whole thing. She's hardly the only corrupt person in D.C. and it's more likely than not if she were to go down, she could take a lot of other people with her on both sides of the isle. As much as the Republicans love talking about how corrupt she is, exposing it probably slits many of their own throats in the process. Elections are basically a trial in the court of public opinion anyways, so making swing voters think Hillary is guilty is effectively just as good as legally proving it.
    • Re: (Score:2, Interesting)

      ... then we still don't know how plausible it is that they reviewed XXXX number of emails in 11 days, after taking months to review 80,000 emails before.

      It's unclear to me why it matters. You're either voting against Trump, or against Hillary. If you're voting against Trump, you will do so even if Hillary is a known axe murdering pedophile who moonlights as an investment banker. You will hope the FBI catches up with her and slaps handcuffs on her the minute she is done being inaugurated and becomes at bes

  • Bingo (Score:4, Informative)

    by PvtVoid ( 1252388 ) on Monday November 07, 2016 @02:27PM (#53231511)

    It would be very easy via automation to tag the emails which are dupes of ones already in the data set.

    Which, apparently, was all of them. No shit, Sherlock.

  • reviewing non-dup emails is hard.

    • by 110010001000 ( 697113 ) on Monday November 07, 2016 @02:32PM (#53231573) Homepage Journal
      There were no "non-dups". There were no new emails. Not too hard to grasp.
      • No, there were new emails. The reason the FBI could easily churn through them was because they were either of a personal nature (and thus not relevant) or didn't involve Clinton at all.
        • by Jhon ( 241832 )

          If by "didn't involve Clinton at all" you mean "did not originate from her" or "was sent to her", then yes. However, she might have been part of relevant chains -- or topics were being discussed by others ABOUT her or what she was told. Those are much harder to sift through automagically.

          Of course -- such emails may not exist in this new batch.

          Whoever wins this election is going to be so damaged that governing is going to be very difficult dipping in to impossible.

          • Comment removed based on user account deletion
            • by ScentCone ( 795499 ) on Monday November 07, 2016 @03:44PM (#53232287)

              because the entire investigation was to check whether Clinton's use of a private server had in some way broken the law

              No, the FBI was looking into whether SHE broke the law(s). And as Comey pointed out in July (and hasn't changed since), she demonstrably did things that would result in any other government employee facing punishment. This isn't about "the server," it's about the double standards. That she mishandled classified information is established. That she lied about it, repeatedly, is established. That she's being held to a different standard is established. Anyone else applying for a high-level, sensitive job in the government with her track record would never, ever be hired (presuming they were out of jail and able to apply in the first place).

            • If it had a disk big enough to hold 'that many' eMails ...

  • He didn't do shit (Score:5, Insightful)

    by darkain ( 749283 ) on Monday November 07, 2016 @02:28PM (#53231525) Homepage

    Snowden didn't do shit. As much as we all "love" him for his previous leaks, he didn't shoot ANYTHING down. He only answered how to dedup a list to make it smaller, not answer how large the list would be after the fact or how long it would take to comb through said remaining list.

    • Re: (Score:3, Informative)

      Um, if you "deduped" the list using hashing the resulting list was zero because there were no fucking new emails. It would literally take less than 5 minutes to run the algorithm.
      • Less than that, I expect. This is hardly earth-shattering work, and the tools to de-dup text files has been around for decades.

        • The headline talks about 650,000 eMails.
          So, no. It would not take a few seconds.
          Not even an SSD is fast enought to randomly access 650k blocks in a second or a few, on a spinning hard disk it would take minutes to read the headers of so many emails, if not hours.
          Such analyzises are a disk (I/O) problem, not memory or CPU.

      • What's 650,000 - 55,000 again?

    • by ceoyoyo ( 59147 )

      Snowden gave a straightforward answer. In technical terms it's pretty trivial. In political terms it's earth shattering.

  • by Kichigai Mentat ( 588759 ) on Monday November 07, 2016 @02:29PM (#53231539) Journal
    Seriously, anyone who's ever had to do de-duplication or pattern matching or anything like that could have told you how easy this is to do. It's almost like computers are good for this kind of stuff!
    • by MightyMartian ( 840721 ) on Monday November 07, 2016 @02:36PM (#53231615) Journal

      The problem is that if de-duping is easy, that means that it could be quickly ascertained if the new mail dump had anything significant in it, which means there was only a brief period of time in which some fantastical new load of Clinton-destroying emails would be found, and if that were the case, then the Trump camp was literally hanging on to a false hope.

      So now we have some of the most tech savvy people on the Internet pretending they're simpering halfwits with know technical know-how at all, just so they can keep a faint hope alive. I guess they can keep imagining Clinton impeachment, though they won't have the votes in the Senate, and it may turn out they don't even have the votes in the Senate to do much else but filibuster Clinton nominees.

      • if that were the case, then the Trump camp was literally hanging on to a false hope.

        Yes, that's exactly the case.

      • by J053 ( 673094 )

        ... it may turn out they don't even have the votes in the Senate to do much else but filibuster Clinton nominees

        And if Chuck Schumer has his act together, he can have the Senate rules changed on day 1 of the new session to eliminate filibuster for any confirmations.

        • Which, if I understand it, is a plan under review if the Senate ends up deadlocked or with a slim Democratic lead. Projections seem to be pointing to 50 Republicans and 50 Democrats+independents, so I think it's likely they'll eliminate filibusters for confirmations.

        • And if Chuck Schumer has his act together, he can have the Senate rules changed on day 1 of the new session to eliminate filibuster for any confirmations.

          Because that would NEVER come back to haunt the liberals later, of course. Be careful what you wish for.

          • Of course it would come back to haunt the Democrats, but the Republicans would share the responsibility. If the Republicans were to back away from their claims that they'd do everything in their power to obstruct the confirmation of Clinton nominees, maybe the Democrats wouldn't use the nuclear option.

          • by Xest ( 935314 ) on Tuesday November 08, 2016 @04:42AM (#53235627)

            As much as you may wish to hope you can stop it the trajectory is towards liberalism, this is why even with the odd hiccup such as Brexit and possibly a Trump victory they're still ultimately only blips on the overall timeline of history. Trump and Farage alike are entirely dependent on people who will be dead in 10 - 20 years to even remotely achieve the numbers they need to reach the goals they want. Beyond that they and their mindsets are well and truly done.

            Liberalism goes hand in hand with intellectualism, as people become better educated on average, more knowledgeable on average, they want more freedom, more rights. They're never going to vote for someone who wants to create interment camps, who calls for political opponents to be assassinated, who hates people over arbitrary and meaningless traits such as sex, sexuality, skin colour and so on. The only way you can stop this tide of change is by making people more stupid, and guess what happens when you do that? you lose the global geopolitical race to someone who hasn't made their population more stupid, and who is progressive, does respect intellect, and in turn pushes human advancement forward with or without you, at which point you adapt and follow or face poverty and irrelevance.

            Human advancement is a basic instinct that no amount of conservatism can put a stop to. Japan and Germany didn't lose World War II because of any particular military strategy, because of bad luck, and so forth, but because when you don't respect intellectuals, those that do get things like the atom bomb instead, and then they win.

            When you understand this, you'll understand why liberalism is such a powerful and effective force that you should probably embrace, rather than continue to fight a war you will never win, as much as a handful of ultimately irrelevant short term victories many excite you.

            This is why liberals have nothing to worry about. They're not losing, and human progress ensures that will always be the case - it's been the overarching trajectory throughout the entirety of human history.

    • by thegarbz ( 1787294 ) on Monday November 07, 2016 @02:50PM (#53231739)

      Seriously, anyone who's ever had to do de-duplication or pattern matching or anything like that could have told you how easy this is to do.

      So not a Trump supporter then. RegEx? That's just a Clinton propaganda piece. Computers too good? They are made in China. Nothing good comes from China. You're just a paid shill standing between us an a great America.

    • Seriously, anyone who's ever had to do de-duplication or pattern matching or anything like that could have told you how easy this is to do. It's almost like computers are good for this kind of stuff!

      This implies that a lot of Clinton-haters on /. don't understand how computers work because they bought Trump's line completely.

      It's almost like when a political opponent is involved, they're willing to ignore clear and obvious explanations in favour of conspiracy theories.

    • by Zocalo ( 252965 )
      Damn right. Snowden has already addressed the de-dupe question so, just for grins, I had a play with my Gmail box to look into the pattern matching. It's got over 400,000 email threads in it containing several million individual emails (I've had it since 2004 when it was still in early beta), yet Google was able to return all the emails that contained either of the search terms "Hillary" and/or "Clinton" in a couple of seconds. That included emails that contained "Hillary" or "Clinton" in the body or in
  • by dave562 ( 969951 ) on Monday November 07, 2016 @02:30PM (#53231549) Journal

    I work for an organization that is heavily involved in electronic discovery processing for large corporations, law firms and the United States government.

    Email threading, and duplication detection / dedupe are standard tasks that are performed on a daily basis on huge datasets. (As part of the Processing phase of the EDRM model.)

    It is not at all unfeasible that the FBI could have used standard, off the shelf software to identify duplicates and generate an exception report for all 'new' emails that were not in the previously collected datasets.

    https://en.wikipedia.org/wiki/... [wikipedia.org]

    • Doesn't MS-Exchange, as an example, do that automatically so that emails sent to multiple recipients on a server only have multiple pointers to one email in the message store?

      • by dave562 ( 969951 )

        That would make sense. I have not seriously administered an Exchange server since 2003 and only had passing familiarity with the application's 2007 version.

        The processing tools do not rely on the server to tell them which emails are duplicates. They do what Snowden suggested. Hash the emails and then compare the hash values.

      • Yes, but each reply is it's own Message-ID, so while each mailbox on a store has a link to a single message, every response contains is a complete item (New Message-ID), not just the response.

    • by sribe ( 304414 )

      Email threading, and duplication detection / dedupe are standard tasks that are performed on a daily basis on huge datasets. (As part of the Processing phase of the EDRM model.)

      Hell, thanks to a massive screw-up with OS X upgrade, I actually needed to check for dups this weekend. A half-hour to write the script, about 30 seconds running time for 250,000 emails on a 5-year-old laptop with an old-fashioned spinning disk--single core only, no need to break the job across cores...

      I would assume the FBI to be vastly better at this than I am ;-)

  • This just in (Score:4, Insightful)

    by Verdatum ( 1257828 ) on Monday November 07, 2016 @02:32PM (#53231571)
    Guy answers high school freshman-level tech support question. We'll have details on this exciting story as they develop.
  • Re: (Score:2, Insightful)

    Comment removed based on user account deletion
    • How would a Republican president, even Trump, be different in curtailing "More global interventionist policies, larger and more intrusive government, and a supreme court that will rubber stamp all of it."

      What has Bush Jr. (the last Rep. prez) , or the republican congress done in the last, say 16 years, done to stop or slow any of these things?

    • by Jeremi ( 14640 )

      Hillary's supporters simply do not care about any of this. They aren't electing Hillary - they are electing an ideology.

      An alternate explanation: Many of them are voting for Hillary specifically because they don't want Trump anywhere near the White House. The guy is truly insufferable, and a clear and present danger to the nation.

  • by Lumpy ( 12016 ) on Monday November 07, 2016 @03:07PM (#53231925) Homepage

    Anyone that has ever used a database knows how easy this is. Sadly team Trump is all about spouting words from the mouth at random, and learning how things are done after the fact.

  • Everybody seems to have forgotten that HRC answered the original subpoena with _hardcopies_. Even if you scanned and OCRed them, their hashes will not match due to inevitable missing metadata and formatting differences, OCR typos, etc.
  • Analysis of a mass of written work for duplicateness is a dumb little initialization task compared to that of analysis for content.

  • we need a 3rd party candidate or a 269 269 tie!

  • Cyber (Score:3, Insightful)

    by dohzer ( 867770 ) on Monday November 07, 2016 @03:34PM (#53232203)

    Snowden is lying. My son is seven. So good at cyber. Cyber so good. China beating cyber. Hashtag winning. Hashtag crooked Hilary. Cyber.

Genius is ten percent inspiration and fifty percent capital gains.

Working...