How the White House lost 5 million e-mails

Late last week, as part of a lawsuit filed by advocacy groups, a federal magistrate judge ordered the White House to search for and preserve all e-mail messages sent and received by employees between March 2003 and October 2005, a period in which more than 5 million White House e-mails seem to have gone missing.

Today Ars Technica's Timothy B. Lee offers a thorough look at the case, explaining that the messages were lost, in part, due to an "upgrade" that Bush administration officials instituted when they came into office.

In 1994, the Clinton White House installed an automatic message-archiving system to work with Lotus Notes, the e-mail software that the White House used then. But in 2001 the Bushies, determined to restore honor and integrity to the White House, scrapped Lotus Notes for Microsoft Outlook.

The Clinton administration's archiving system wouldn't work under Outlook, so according to a congressional report (PDF), Bush officials instituted a new archiving plan in which a "White House staffer or contractor would collect from a 'journal' e-mail folder in the Microsoft Exchange system copies of e-mails sent and received by White House employees."

In other words, they dropped automated archiving in favor of manual archiving. Seriously, they did this.

Lee, quoting Steven McDevitt, a former White House IT official who quit in 2006, notes:

Because the archiving process was conducted manually and in an ad hoc fashion, human error could easily lead to the inadvertent omission of e-mails that are required to be preserved under federal law. Files were "scattered across various servers" on the network of the Executive Office of the President, and there "was no consistently applied naming convention" for the files. It's hardly surprising that things tended to get lost.

Even more troubling, due to a lack of redundancy and proper access controls, anyone with access to the White House servers could have tampered with or deleted the e-mails in the archives. And without adequate logging facilities, there might be no way to determine who might have tampered with the files or what might have been changed.