Stories
Slash Boxes
Comments

News for nerds, stuff that matters

Slashdot Log In

Log In

Create Account  |  Retrieve Password

Whitehouse Emails Were Lost Due to "Upgrade"

Posted by CmdrTaco on Wed Apr 30, 2008 09:48 AM
from the upgrading-to-orwell dept.
I Don't Believe in Imaginary Property writes "We now know how the Whitehouse managed to lose about five million emails. It seems that they 'upgraded' their Lotus Notes system, which had an automatic retention and backup system, for Microsoft Exchange, which did not support the automatic system. So they changed it to a manual process, where aides would manually sort emails one by one into individual PST files, which they call a 'journaling' archive system. They're still building a replacement for the retention system. Right when they had one finished, the White House CIO complained that it made Microsoft Exchange too slow, so they hired yet another contractor to build another one, causing a senior IT official to quit in protest. So they still haven't completed the project after almost eight years, and rely on humans to sort millions of emails."
+ -
story

Related Stories

[+] 22 Million Missing Bush White House Emails Found 326 comments
ctmurray writes "Computer technicians have found 22 million missing White House e-mails from the administration of President George W. Bush, and the Obama administration is searching for dozens more days' worth of potentially lost e-mail from the Bush years, according to two groups that had filed a lawsuit — which has now been dropped — over the failure by the Bush White House to install an electronic record-keeping system. Earlier we discussed the Obama White House's opposition to the lawsuit that led to this discovery." The related links reflect our discussions about the missing emails over two years.
This discussion has been archived. No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More
Loading... please wait.
  • by FuzzyDaddy (584528) on Wednesday April 30 2008, @09:50AM (#23250710) Journal
    "Strategic Incompetence"
    • by PawNtheSandman (1238854) on Wednesday April 30 2008, @09:56AM (#23250814)
      "Mission Accomplished"
    • by Jerf (17166) on Wednesday April 30 2008, @10:40AM (#23251434) Journal
      So, today the Bush administration is brilliant, and they came up with a genius plan to make email go away, while appearing incompetent.

      Someone be sure to send me the talking points when we're back to "The Bush administration is staffed by morons", k?

      (Such amazing IQ swings we see. Genius! Moronic! Brilliant! Ape-like! Bing-bam-boom! Sometimes several flip-flops in one day! One would almost wonder if the problem lies in the observers, rather than the observed.)

      "Never attribute to malice what can be adequately explained by incompetence." I think "incompetence" covers it just fine; I'm sure this is hardly the first migration screwed up this way.
      • by Chris Burke (6130) on Wednesday April 30 2008, @01:33PM (#23253734) Homepage
        (Such amazing IQ swings we see. Genius! Moronic! Brilliant! Ape-like! Bing-bam-boom! Sometimes several flip-flops in one day! One would almost wonder if the problem lies in the observers, rather than the observed.)

        The problem lies in this ridiculous line of thinking where someone can only ever have one adjective applied, and that adjective must apply to everything they do.

        Here's the dope: The Bush White House is quite adept at playing politics -- genius when Rove was involved -- including yes the ability to make apparent incompetence into a strength. They are skilled at making the organizations they control work for them, producing the information they want to hear, and failing to find or losing the information they don't want anyone to hear, to support their political goals. When it comes to actually executing policies outside of Washington, they're terrible failures because in reality you can't get rid of facts you don't like and keep only the ones you do.

        What's so contradictory about that? I'm "brilliant" with computers, I'm "moronic" with cars. To think that one precludes the other is idiotic. But then again, so is the whole "flip-flop" figure of speech.
      • by Thing 1 (178996) on Wednesday April 30 2008, @02:07PM (#23254198) Journal

        So, today the Bush administration is brilliant, and they came up with a genius plan to make email go away, while appearing incompetent.

        Well, yeah, except: what happened to the final backup tapes of the first installation?

        Since it's the last backup of that system it should definitely be marked for retention. And surely, as they realized that they had a retention issue with the new system, they would have ensured to maintain those tapes due to the Presidential Records Act that Bush himself amended?

        Also, doesn't it concern anyone that he changed the law regarding what communications can be released and when on Nov 1 2001, [fas.org] just three weeks after 9/11? Coincidence and circumstantial, perhaps, but concerning...

        • Re: (Score:3, Insightful)

          You sir are on the mark.

          You can get a long line of IT admins from around the country to testify how big the lies coming from the Administration are, put the white house IT admins on the stand and rip them to shreds, then throw their asses in JAIL when they show gross incompetence in following the law, instead of coming right out with the truth of what happened and who encouraged it to happen. Plausible deniability only succeeds when noone has the balls and patience to search for the truth. This is not som
  • by zappepcs (820751) on Wednesday April 30 2008, @09:50AM (#23250714) Journal
    read the summary and understood the Whitehouse is blaming Microsoft? hahahahahahahahahahahahahahahaha

    There is nothing that will happen for the rest of the week that can make me more light hearted than this. HAHAHAHAHAHAHAHAHA
    • Or that the aides are manually sorting emails. What a crappy job.

        • Re: (Score:3, Informative)

          I think that's what they did. They turned on journaling, and then archived the journaling account to PST files.

          Unfortunately, this meant that a person was manually copying to PST, which introduces an opportunity for either human error or tampering. In addition, PST files aren't very good for this sort of archive. They've long had a history of getting corrupt as they grow in size, they're hard to search, and they don't have much in the way of built-in security controls. It'd be better to dump the files i

  • These days... (Score:5, Insightful)

    by neokushan (932374) on Wednesday April 30 2008, @09:52AM (#23250740)
    It's getting harder and harder to tell the difference between subterfuge and sheer incompetence.
    • These days? (Score:5, Interesting)

      by wiredog (43288) on Wednesday April 30 2008, @09:54AM (#23250784) Journal
      Napoleon Bonaparte is credited with saying "Never blame on malice that which can adequately be explained by incompetence."
      • Re:These days? (Score:5, Insightful)

        by Shinmizu (725298) on Wednesday April 30 2008, @10:00AM (#23250864)
        And any sufficiently advanced incompetence is indistinguishable from malice.
      • Re:These days? (Score:4, Insightful)

        by Danse (1026) on Wednesday April 30 2008, @10:14AM (#23251086)
        Once you get to a certain level of incompetence, it's really indistinguishable from malice. In this case, the incentives are all there for them to want to keep this "problem" in place. It lets them conveniently lose any incriminating email and blame it on "them dang computers". Everyone's lost some files at one time or another, right? Ok, so maybe you didn't have your own IT department in charge of running the communications for the most powerful government in the world...
      • Re:These days? (Score:4, Insightful)

        by clem (5683) on Wednesday April 30 2008, @01:30PM (#23253702) Homepage
        Though malice and incompetence are not mutually exclusive.
    • Re:These days... (Score:5, Insightful)

      by Frosty Piss (770223) on Wednesday April 30 2008, @10:02AM (#23250908)

      It's getting harder and harder to tell the difference between subterfuge and sheer incompetence.
      Because the bullshit is so deep?
  • by utnapistim (931738) <dan...barbus@@@gmail...com> on Wednesday April 30 2008, @09:53AM (#23250760) Homepage

    What was that quote about never ascribing to malice?

    It's a well put-together story (plausible enough) but I'm still skeptic though.

    Maybe we've just seen too many lies :)

  • Six P's (Score:3, Interesting)

    by gregarican (694358) on Wednesday April 30 2008, @09:55AM (#23250794) Homepage
    Prior Planning Prevents Piss Poor Performance. This shouldn't be taken as a Micr$oft bash as much as an example of poor planning. After having administered both Lotus Notes and Microsoft Exchange I can say that ditching Notes for Exchange isn't a bad move. But doing so without planning out the migration path is. Any large scale project should involve a considerable outside contracting firm that would have automated measures in place. You could even plug in a server appliance before your front end Exchange servers that would automatically archive off mail messages being sent to/from the White House staffers. Another example of US government being inept. Just look to how the US air traffic control centers still operate with equipment that is so outdated that some units are out of commission because they can't order vacuum tubes to service them...
    • Re:Six P's (Score:5, Insightful)

      by Jeremi (14640) on Wednesday April 30 2008, @10:09AM (#23251000) Homepage
      This shouldn't be taken as a Micr$oft bash as much as an example of poor planning.


      Or perhaps an example of really good planning. If I was planning to make sure a few million potentially incriminating emails never found their way into the public eye, that is how I might do it. Certainly if I had spent a number of meetings discussing how and when Americans should torture people [washingtonpost.com] I would be motivated to do so.

  • by roystgnr (4015) <roystgnrNO@SPAMticam.utexas.edu> on Wednesday April 30 2008, @09:55AM (#23250796) Homepage
    The White House's failure to follow records retention laws was due to deficiencies in Microsoft software?

    I predict this will lead to a civil, thoughtful Slashdot discussion which results in many useful recommendations for avoiding similar problems in the future.

    I recommend fire.
    • Re: (Score:3, Funny)

      The White House's failure to follow records retention laws was due to deficiencies in Microsoft software?

      And it was completely unnecessary as Notes will happily set emails to zero length, without warning, if you archive them to a drive with no remaining space. No Microsoft products needed!

    • Re: (Score:3, Informative)

      The part that I find difficult to swallow is that they decided to change to Exchange knowing that it didn't meet the retention requirements and knowing that they've have to have aides sorting through millions of e-mails. I can't even imagine the kind of thinking that allowed that to happen, other than to make a *cough* "plausible" *cough* case for e-mail going missing.

      Still, how did such volumes of e-mail actually disappear? Either aides were sorting all e-mails into individual PST files and thus all the e-
  • by owlnation (858981) on Wednesday April 30 2008, @10:01AM (#23250894)
    Basically this comes down to either:

    The Government was Incompetent.

    or...

    The Government is lying and covering up.

    Hmmm.... Mr Rock, meet Mr Hardplace.
    • by ColdWetDog (752185) * on Wednesday April 30 2008, @10:09AM (#23251002) Homepage

      Basically this comes down to either:

      The Government was Incompetent.
      or ...
      The Government is lying and covering up.

      I understand the concept of Occam's Razor, but it could really be a case of AND instead of OR.

      It fits the results better, actually.

  • by Reality Master 201 (578873) on Wednesday April 30 2008, @10:02AM (#23250914) Journal
    Seriously, this is the least bullshit excuse the could come up with? If ANY corporation in the US tried this kind of thing, the wrath of SARBOX would rain down on them like you wouldn't believe.

    Even given the staggering incompetence of the Bush administration in nearly all aspects, this just doesn't pass the laugh test.
  • by metoc (224422) on Wednesday April 30 2008, @10:03AM (#23250926)
    1) To bad the Whitehouse isn't using an e-mail system like millions of other people. Wait they are. Like it or not MS Exchange is everywhere.
    2) To bad the requirement for e-mail archiving and retention is unique to government. Wait, most publicly traded companies have legal and compliance requirements to do so.
    3) To bad there is no market for software to archive and retain e-mail on one of the most common e-mail platforms. Wait, there is, and its huge.
    4) To bad nobody has nobody has developed technology for this market. Wait, there are dozens of solutions.

    To bad no one is getting fired, imprisoned or impeached over this one.
  • by damn_registrars (1103043) on Wednesday April 30 2008, @10:06AM (#23250968) Journal
    ... to how the Bush white house oversees "job creation":

    rely on humans to sort millions of emails
  • by OpenSourced (323149) on Wednesday April 30 2008, @10:08AM (#23250990) Journal
    and rely on humans to sort millions of emails.

    No problem. They had the job outsourced to India.

  • bloody hell. (Score:4, Insightful)

    by apodyopsis (1048476) on Wednesday April 30 2008, @10:14AM (#23251092)
    utter bollocks. just unbe-fucking-lievable.

    *every* backup system should result in a set a of data offsite or in a storage area never to be touched again.

    even if you use incremental backup every nth backup should be a complete archival read only copy re the previous sentence.

    the *very* worst case should be the last major backup is in a format that is not readable with the current system and some red faced admins need help to read read the data.

    5 million emails? jesus wept.

    add the conspiracy theory factor into the mix and you have something that, on the face of it, sounds unbelievable.

    as one of our politicians in the UK said to another a short while ago "you cannot have it both ways, you were either ignorant or incompetent - and neither is acceptable".

  • Criminal? (Score:4, Interesting)

    So, assuming for a moment their story is true and it *is* just negligence, incompetence, and stupidity; it is still FEDERALLY CRIMINAL negligence, incompetence, and stupidity, Right? Books will be thrown at those responsible, yes?
  • by ashitaka (27544) on Wednesday April 30 2008, @10:23AM (#23251230) Homepage
    One of my first projects after moving to Vancouver was a couple of test installations of pre-release Exchange server back in 1996. Since then I've worked constantly with every version of Exchange in all kinds of backup situations. Early versions of Exchange were a bitch to restore but it's gotten better.

    However, there has *always* been a way to retain and archive emails automatically from Exchange and no shortage of migration utilities from notes to Exchange. The reasons stated in the article just don't wash. No one, not even the newest tech school grad could come up with a system like that currently in use.

    However, it may in fact not be intentional malice from the start but more likely an existing state of incompetence that was taken advantage of to hide traces or misdeeds or at least to make finding any evidence difficult.

    This still doesn't address the use of non-government email systems for official business by Rove and other Republican members. According to the laws of the United States this is all highly illegal. Don't you care at all about what your government is doing or do you think whatever you do won't make any difference?
  • They are still lying (Score:4, Informative)

    by Ralph Spoilsport (673134) * on Wednesday April 30 2008, @10:30AM (#23251318) Journal
    everything that goes through the WH is:

    a: saved to tape and sent to a vault on a daily basis
    b: recorded by the NSA, who also saves and backs up data

    So, it's all a load of bullshit - they're thinking that the public is stupid enough to buy it, or, simply kick it down the road another month or two until the ADHD press finds something shiny to get distracted by like Miley Cyrus Boobs or another blast from Trainwreck Spears.

    RS

  • by Sleepy (4551) on Wednesday April 30 2008, @11:03AM (#23251766) Homepage
    The industry is full of stories like this, for years. Exchange by default can't handle it - it's still a workgroup server at heart, and subject to many OS and filesystem limits. Does Hotmail.com even rely 100% on Exchange, or is it still UNIX at the core?

    De-centralized email storage and PST files?? COME ON!

    It is almost CERTAIN to expect that they knew this would cause emails to be lost and take the system from bad to worse. Even a junior IT person fresh off the boat would say this was CRAZY to attempt, with FEWER benefits and increased risk. In the corporate world, this would be met by massive civil lawsuits and possibly criminal charges. Any "contractor" the WH employed would know this for a fact.

    So given that such warnings had to have been given and they went ahead anyways, you have to wonder if strategic "loss" of emails was perfect cover for an email purge. Given the shady nature of these characters, I'm sure this was a calculated "feature".

    • Re: (Score:3, Insightful)

      ridiculous!
      Evidently, you have never used Lotus Notes!
          • Wait, does that mean a Microsoft product is actually better?

            A friend of mine used to work for IBM. They (his department, at least) used Outlook.

            If that doesn't say it I'm not sure what does. I've heard some probably justified horror stories about being the person who needs to admin the Exchange server, but from the perspective of a normal user who just wants to read their e-mail, schedule meetings, etc... Outlook is ridiculously better. (Or was. I haven't used the latest major version of Notes.)
      • Re:yes it is. (Score:5, Insightful)

        by SatanicPuppy (611928) * <(Satanicpuppy) (at) (gmail.com)> on Wednesday April 30 2008, @09:55AM (#23250802) Journal
        Not to defend microsoft, but COME ON! Who do they have doing their tech support? Is Bush doing it himself?

        I find this frankly impossible to believe, and insulting on top of that.
        • My wife works for USDA. On several occasions, she has gone a week or more without email due to botched Lotus Notes upgrades. Mostly, I think it was due to incompetent contractors, but, considering the times I've had a gun put to my head and forced to use Notes (over a slow WAN connection - the very definition of torture) I am sure some of that is intrinsic to the application. I'm not sure what they use now, she does 90% of her email on her blackberry, but I think they have gone to Exchange/Outlook.
        • No, he suggested that Microsoft patriotically gained access to electronic communications in an effort to prevent terrorism. If the subjects have nothing to hide, what do they fear from Microsoft's oversight?
      • Don't underestimate the incompetence that can be found in the corporate world.

        You too can be a CIO! Just learn how to play golf, join the right country club, and let one of Microsoft's sales representatives nibble on your sweet, sweet braaiinnnsss!

    • by Woundweavr (37873) on Wednesday April 30 2008, @10:04AM (#23250930)

      why throw out what works?

      Because "works" in this case is a means by which they can get caught?

      If I was going to be as corrupt/incompetent as this administration, I'd try to limit how much that criminality/idiocy could be directly documented for criminal proceedings/historical study.
    • by pclminion (145572) on Wednesday April 30 2008, @10:57AM (#23251684)
      How is incompetence an excuse for violating the Presidential Records Act?
      • by ardent99 (1087547) on Wednesday April 30 2008, @11:05AM (#23251792)
        I have hard time believing it is not intentional incompetence or malice. But at this point it doesn't matter. They have been committing crimes by not keeping a record of government email, and by using GOP computers to handle whitehouse email, and by not supplying the records to Congress after being subpoenaed, and by ordering the Justice Department not to enforce the subpoena. This is a massive cover up on a huge scale, and they have managed to block every attempt by Congress to investigate it. Whether they are covering up malice or negligence doesn't make the cover up legal.

        However, the reason it must be malice (aka intentional incompetence), is that for these purposes it doesn't matter whether the files were correctly converted from one format to another. They could have given over all the records to Congress in whatever disordered form it was in, and let Congress figure out how to sort through it. There are very easy ways to pull information out of a complete morass of files. For example, just text index the whole mess, and search for any text containing interesting phrases, and then follow the references in those text blocks to related text blocks. You could probably get 90% of the meaning of a collection of email in random formats just by doing things like that. But obviously the whole point is to block that from happening, not enable it.
    • Re: (Score:3, Informative)

      I call FUD. Exchange supports POP3, IMAP, etc. Enabling/disabling it is trivial.

      With versions of Exchange prior to 2007, it was trivial to export an entire mailbox directly from the Exchange store (and reimport it later). I don't know why it would be necessary to delete an Exchange mailbox like that to fix a problem, but at the very least you could have copied everything from within Outlook to a local .PST. This would have saved everything except your rules.

      There are, and have always been, many good way
    • "Upgrading our Lotus Notes with Microsoft Exchange allowed White House staff to cut jail time by more than 83%."
    • If your TCO without Microsoft products losing your data includes jail time then I think that's pretty compelling evidence that Microsoft does lower your TCO...