CASE MANAGEMENT ORDER NUMBER 63 Regarding Microsoft Upgrade
DAVID R. HERNDON, Chief District Judge.
I. INTRODUCTION
This matter is before the Court on a pending issue related to the defendants' systems upgrade from Windows XP to Windows 7/One BI Client for all BII and BIPI custodians subject to the litigation hold.1 Prior to addressing the matter presently before the Court, the Court will provide a brief background.
II. RELEVANT BACKGROUND
On October 18, 2013, the defendants informed the Court regarding the loss of material from Prof. Dr. Klaus Dugi's (the defendants' Head of Corporate Medicine) laptop (Doc. 311-4). Specifically, the defendants reported as follows:
Defendants learned that Prof. Dugi's laptop was not backed-up successfully in August 2013 when the operating system was upgraded to the OneBI Client (from Window XP to Windows 7) as part of a company-wide migration and software upgrade. Because Prof. Dugi was subject to a Litigation Hold, the IS department backed up the data on his laptop to his user share while the computer's operating system was upgrading to the OneBI Client. After the upgrade was complete, IS attempted to restore the data, but was not completely successful. BII then retained e-discovery forensic expert, Kroll Ontrack, to examine and restore the hard drive but their efforts were unsuccessful as well. As a result, the re-sweep of Prof. Dugi's laptop will not include data stored there between late December 2012 and August 2013.
(Doc. 311-4 pp. 2-3). The defendants additionally informed the Court that to prevent a similar occurrence, "the migration to Windows 7/OneBI Client of any BIPI or BII custodian subject to a Litigation Hold has been stopped" (Doc. 311-4 p. 3).
On February 20, 2014, the defendants informed the Court that "as a matter of business necessity, [the defendants] have recommenced upgrading the operating systems on the laptops of BII personnel." See Exhibit A p. 1. The defendants further explained that after April 8, 2014, Microsoft would no longer support Windows XP. Id. Accordingly, the defendants informed the Court, in order to continue business operations, the defendants' needed to recommence upgrading employees' operating systems. Id.2 On February 27, 2014, the PSC wrote to the Court expressing concern over the upgrade. See February 27, 2014 Letter to the Court (attached hereto as Exhibit B). The PSC indicated that it was concerned about the upgrade in light of previous failures, including the lost information on Prof. Dugi's laptop. Id. The defendants responded to the PSC's objections via letter dated February 28, 2014. See February 28, 2014 Letter to the Court (attached hereto as Exhibit C).
After hearing argument on the matter at the status conference on March 5, 2014, the Court ordered that the defendants create and preserve for the duration of the litigation a mirror image of the hard drive of all custodian laptops scheduled for the Microsoft upgrade from Windows XP to Windows 7 (Doc. 440 p. 15 ll. 1-18).
III. MATTER PRESENTLY BEFORE THE COURT
On March 19, 2014, the defendants wrote to the Court seeking reconsideration of the Court's March 5, 2014 order pertaining to mirror image copies of custodian hard drives. See March 19, 2014 Letter to the Court (attached hereto as Exhibit D). The defendants advised that creating the mirror images would lead to storage problems and result in other business inefficiencies. Accordingly, in lieu of creating mirror images, the defendants requested permission to preserve by storing the existing hard drives, rather than upgrading them. Id. Instead of receiving an upgraded hard drive, the subject employees would be provided with a new upgraded hard drive and the existing hard drives would be stored for the remainder of the litigation. Id. Subsequently, at the direction of the Court, brief letter briefs were provided by the PSC and the defendants. See March 21, 2014 Letter to the Court (attached hereto as Exhibit E), March 24, 2014 Letter to the Court (attached hereto as Exhibit F), and March 25, 2014 Letter to the Court (attached hereto as Exhibit G).
The PSC insisted that mirror imaging was the best and most reliable way to ensure that, in the event of a technical issue with an upgrade, the parties would be able to recover the entirety of the information on the laptop pre-upgrade. The defendants insisted that storing the existing hard drives was an acceptable approach to the upgrade and the most appropriate alternative given the alleged hardship preservation via creating mirror images would impose on the defendants.
IV. CONCLUSION
After reviewing the parties' arguments and the affidavit provided by the PSC in their March 25, 2014 letter to the Court, the Court finds that the cost benefit analysis weighs in favor of the defendants position. Accordingly, the Court ORDERS as follows:
The defendants' request to preserve by storing the existing hard drives for the duration of the litigation in lieu of creating and storing a mirror image of the hard drives is GRANTED. FURTHER, the defendants are ORDERED to place the hard drives in a storage facility that is environmentally conducive to the continued viability of the integrity of said hard drives based on universally accepted computer industry standards.
SO ORDERED:
EXHIBIT A
UNITED STATES DISTRICT COURT
SOUTHERN DISTRICT OF ILLINOIS
In Re Pradaxa (Dabigatran Etexilate) Products 3:12-rnd-02385-DRH-SCW
Liability Litigation MDL No. 2385
This Document Applies To All Actions
AFFIDAVIT OF BRUCE PIXLEY
County of Sacramento )
)
State of California )
Bruce Pixley, being duly sworn deposes and says:
1. I am currently the principal of Pixley Forensics Group. I have more than 25 years of combined experience in computer forensic analysis, high-tech investigations, and law enforcement. I am a retired lieutenant in the Santa Barbara Sheriff's Department and was responsible for the creation and implementation of the county's first High Tech Crime Unit and computer forensics' lab in 1999. Since that time, I have been involved in both the imaging of hard drives and overseeing large scale projects involving the imaging of over one hundred hard drives.
2. Since 2001, I have served as a lead instructor of computer forensics, Internet investigations, and network intrusion courses for the California Department of Justice's Advanced Training Center. Additionally, I have been employed as a Master Instructor at Guidance Software, which developed the EnCase computer forensic software. As an instructor, I have taught for over 2,000 hours on the subjects of computer forensics and high-tech investigations. Additionally, I have developed course training materials and wrote manuals for computer forensic courses such as Advanced Internet Examinations and Network Intrusion Investigations.
3. I possess three professional certifications for my fields of work. I possess the Certified Information Systems Security Professional (CISSP) certification and the GIAC Certified Forensic Analyst (GCFA) certification, which are both ANSI ISO accredited credentials, and the EnCase Certified Examiner certification.
4. Since 2003, I have been retained as a computer forensic examiner and subject matter expert in both criminal and civil matters. I have been qualified as an expert witness in both state and federal courts and testified about the foundation of computer forensics, Windows and Mac operating systems, chat software, Internet and network operations, e-mail, peer-to-peer file sharing, digital photography, recovery of deleted data, and Trojan viruses.
Documents Reviewed and Summary
5. I have reviewed the transcript of the status conference hearing on March 5, 20,14 and the Defendant's letter to the court dated March 19, 2014.
6. A mirror or forensic image ("image") of a hard drive serves two purposes: 1) Preservation, as it is a complete and verifiable sector-by-sector copy of the original hard drive for evidentiary purposes; and 2) Storage Efficiency, as images of multiple drives may be stored on a single hard drive.
Storage Efficiency and Data Integrity
7. When creating an image of a hard drive, the actual image is often compressed to decrease the amount of storage space required. This function was specifically designed for storage efficiency purposes. For example, the image of an 80 gigabyte hard drive may easily compress down to-less than 30 gigabytes. This means that in a corporate environment, you could store over 60 images on a single 2-terabyte hard drive, which costs approximately $100.00. As more drives are imaged, the efficiencies increase.
8. Whether the forensic images are stored on individual hard drives or a separate networkbased storage device, the storage of the forensic images should not interfere with the normal course of business. In order to prevent interfering with the other business of the organization, the forensic images can be easily kept in a segregated storage environment.
9. The process of imaging multiple hard drives at the same time is also an efficient process as the forensic collection specialists can. setup an assembly-line approach to streamline the entire process. By using basic project management skills, the process can easily track and document the hard drive from the time it is removed from the computer, the actual imaging process, and the hard drive being restored to the original computer. For example, during a imaging process of over 100 hard drives for litigation purposes which I supervised, a team of 3 forensic examiners successfully handled this type of collection without any difficulty or creating business inefficiencies.
10. When creating an image of a hard drive, it is standard practice to create a backup copy of the image at the same time, which is then stored on a separate storage drive. This protects against the possibility of storage drive failure. Additionally, at the time of creating the primary and backup copy of the image, the images are verified to ensure their integrity. This same type of integrity cheek may be used in the future to demonstrate that no one has tampered with the image.
11. During the imaging process, documentation needs to be created to track the image, the source computer/custodian, the hard drive, the storage media and the location of where the storage media itself is being stored. This documentation also provides a chain of custody-for authentication purposes.
Data Risk and Failure
12. It is my understanding that the defendants have proposed to remove the original hard drives from the computers and place the bare hard drives into storage until they are needed, The storage of bare hard drives proposes a risk to the data stored on the drives and becomes a single point of failure for the following reasons:
a. Hard drives that have been in service for years have a greater chance of drive failure. Eventually these hard drives will no longer spin and the data on the drives may not be recoverable.
b. If a drive is accidentally dropped or otherwise damaged (e.g., by flood, heat, etc.) while in storage, there is a very good chance that the data on the drive will not be recoverable,
13. The storage of bare hard drives will require the same typo of documentation that is required in the imaging process. Someone will need to track each hard drive along with the source computer/custodian information for chain of custody and authentication purposes.
14. Unlike an image, a bare hard drive does not have a mechanism in place to"verify that no data on the drive has been altered since the time it was secured as evidence.
Data Extraction
15. If any data needs to be extracted from au image, the image does not need to be restored. Commercially available software already exists to extract that data, including deleted files, directly from the image.
16. Extracting data from images is also an easy task since the multiple images may be kept in the same location, such as a single hard drive or network-based storage. This is more efficient than having to attach separate physical drives for each custodian.
Conclusion
17. Due to the risks involved of storing bare hard drives removed from the custodians' computers, it is my opinion that creating an image of the drive as I described above would be the most efficient and safest method to collect and preserve custodian data.
Further affiant sayeth not.
