CHERYL A. EIFERT, Magistrate Judge.
On April 3, 2015, the Court denied Ford Motor Company's ("Ford") motion for a protective order that would have prohibited Plaintiffs from discovering Ford's Electronic Throttle Control ("ETC") system source code for various makes and models of Ford vehicles.
Since the Court's April 3, 2015 order, the parties have been unable to agree on a protective order governing the production of Ford's ETC source code, and instead, have submitted differing proposed protective orders. During a telephonic discovery conference on April 29, 2015, the parties informed the Court that they principally disagreed over the format in which Ford was to produce the ETC source code. Plaintiffs requested the code in "write-access"
The Court has thoroughly considered the arguments of the parties and
These cases involve alleged events of sudden unintended acceleration in certain Ford vehicles manufactured between 2002 and 2010. In particular, Plaintiffs claim that their vehicles were equipped with defective ETC systems which were not fault tolerant, resulting in open throttle events during which the drivers of the vehicles lacked the ability to control the throttles. Plaintiffs assert that the mechanisms causing the throttles to open unexpectedly were numerous, including electromagnetic interference, resistive shorts, and other voltage and resistance fluctuations, and that these issues were known to Ford. Despite having knowledge of the potential for sudden unexpected acceleration, Ford nonetheless failed to properly design the ETC system to correct the events when they occurred, and further neglected to install fail-safes, such as a Brake Over Accelerator system, that would allow the drivers to physically prevent or mitigate sudden acceleration.
In the course of discovery, Plaintiffs requested that Ford produce for inspection and review the source code for the ETC system. Ford objected, and the parties were unable to resolve the issue in informal discussions. Ford then filed a motion for protective order, asking the Court to prohibit disclosure of the source code in its entirety. The Court denied Ford's motion and ordered the parties to meet and confer regarding the terms for production of the ETC source code under a special protective order. The parties were not able to agree on the terms of a special protective order, and instead submitted proposed protective orders and briefs outlining their respective positions. After discussing the issue no less than twice during telephonic discovery conferences, the Court held a hearing on the issue where testimony was provided by two expert witnesses—Nigel Jones for Plaintiffs and Dr. John Kelly on behalf of Ford.
Mr. Jones earned a Bachelor of Science degree in Electrical and Mechanical Engineering from Brunel University in London, and he is currently employed at the Barr Group as Chief Engineer and at R.M.B. Consulting, Inc., as its president. (ECF No. 496-1 at 1-3). In these positions, Mr. Jones has designed both hardware and firmware for various devices.
After explaining the usefulness of a debugger, Mr. Jones demonstrated how code may be analyzed using a software simulator and the process of fault injection.
Dr. Kelly testified that he is a software engineer and that he has experience working with nuclear reactor safety shutdown systems, flight control systems, medical devices, radiation devices, train control systems, and diesel engine control systems. (Id. at 81, 92). He stated that he had been hired as an expert in a number of cases "to assess and analyze" source code. (Id. at 81). Dr. Kelly asserted that he had "signed onto many protective orders" in some of those cases and that Ford's proposed protective order for producing its ETC source code was "entirely consistent" with those past protective orders. (Id. at 82). According to Dr. Kelly, Plaintiffs are "looking for conditions that would cause the software to fail," which can be accomplished by examining the software, developing a hypothesis, and then "actually testing the real binary on the real PCM [Powertrain Control Module] to see if that can occur." (Id. at 83). Dr. Kelly explained that such testing could be performed on the hardware module, which is connected to a computer, by forcing inputs into various input pins through the application of voltages to specific places on the hardware module. (Id. at 84). He insisted that write access to the source code was unnecessary to perform the testing that Plaintiffs desired and that there is no need "to change the code in order to test the binary." (Id. at 83-84). In response to Mr. Jones's demonstration where he changed certain portions of the code, Dr. Kelly testified that the "numbers set in the [binary] code cannot change" in the real world. (Id. at 84).
In relation to whether a read-only format would obstruct Plaintiffs' experts' ability to analyze the source code, Dr. Kelly asserted that the integrated development environment utilized by Mr. Jones could still be used with read-only access to the source code, as could a compiler and a debugger. (Id. at 86, 98, 100). Dr. Kelly also testified that read-only source code could be "put out on a simulator, as an emulator," and be tested.
On the subject of efficiency in analyzing the source code, Dr. Kelly opined that having the source code in write-access format would not decrease the amount of time necessary to perform most of the tasks that Mr. Jones demonstrated. (Id. at 99). Dr. Kelly remarked that the only process made easier with write-access source code would be testing the code by "forc[ing] it to have certain inputs," which could also be done with black-box testing or white-box testing.
Before the hearing concluded, the parties discussed how the source code would be viewed by Plaintiffs' experts. Plaintiffs' counsel suggested that a physically secure room containing Ford's server be created by Ford wherever it likes and that a virtual private network ("VPN") be established between Ford's server and another location near Plaintiffs' experts. (ECF No. 531 at 112). The location near Plaintiffs' experts would contain computers for Plaintiffs' experts' use that would access Ford's source code and any tools used to analyze the source code over the VPN. (Id. at 113). To access the computers at the Plaintiffs' experts' location, Plaintiffs' counsel suggested that the computers be equipped with biometrics scanners (e.g. fingerprint or retina scanners). (Id.) Plaintiffs' counsel averred that any increased risk that Ford's source code might be intercepted by a hacker who accesses the VPN was "not severe enough" to justify the cost and additional time that would be necessary if all of the source code review were to take place at a facility designated by Ford. (Id. at 114). In response, Ford's counsel objected to Plaintiffs' counsel's VPN proposal and asserted that Ford's source code has never been placed anywhere outside of a secured location at a Ford facility and that the code did not "get carried around or put up on other networks." (Id. at 115, 119); see also (ECF No. 502 at 2) (Ford's brief on the issue stating that Ford has never produced its ETC source code in the manner proposed by Plaintiffs). Yet, Ford's counsel acknowledged that pieces of the source code during the design stage might be transported out of Ford's facility. (ECF No. 531 at 119-20). After hearing the parties' positions, the Court made the parties aware that it shared Ford's concerns as to Plaintiffs' counsel's VPN proposal.
In its brief filed before the hearing, Ford contends that Plaintiffs should only be permitted to inspect and search a read-only version of the ETC source code, and that during Plaintiffs' review of the source code, they should not be allowed to use "additional tools or applications," including compilers and simulators.
In their brief submitted before the hearing, Plaintiffs contend that they must have write access to the source code in order to perform an adequate and efficient analysis of the code. Plaintiffs claim that they must be able to test the source code by injecting faults "into Ford's system to see how it handles or addresses those faults." (ECF No. 504 at 2). They assert that "[t]his is a common and critical testing mechanism." (Id.) Plaintiffs also maintain that they must be able to analyze the "interactions between code sections," and that this cannot occur just by reading the code. (Id. at 4). In addition, Plaintiffs insist that any changes that they make to the source code during their testing of the code can be revealed by using a file comparison tool. (Id.) Finally, Plaintiffs argue that they must be permitted to use a compiler and any related tools in order to "gain a full understanding of how the system actually communicates," and that if they do not have access to a compiler, then the source code review process will be protracted and more expensive. (Id. at 7-8).
After the hearing, Plaintiffs submitted to the Court a spreadsheet containing the names and descriptions of approximately fifty software tools that they hope to use in analyzing the ETC source code. Plaintiffs also indicated whether those tools were compatible with a read-only version of the source code. Plaintiffs' tool list includes virtual machines for operating systems; compilers, linkers, assemblers, simulators, debuggers, integrated development environments, static analysis tools, scripts, and batch files used by Ford; custom tool development programs; script development software; searching and matching programs; word processing programs; Adobe Acrobat; file compression programs; an encryption program; Linux editors; Windows editors; version control software; static analysis tools other than those used by Ford; file comparison software; a metrics tool; and source code "formatters and beautifiers." According to Plaintiffs, a majority of these programs can be used with a read-only version of source code, including the simulators and debuggers used by Ford. In contrast, Plaintiffs assert that the compilers used by Ford "probably" require source code in a write-access format. Nearly all of the tools listed by Plaintiffs would require write access permission for the
In response, Ford's counsel sent a letter informing the Court that Ford objected to Plaintiffs' experts' use of any compiling tools on Ford's ETC source code. In the event that the Court allows Plaintiffs' experts to utilize a compiler, Ford has requested that Plaintiffs' experts' compiler be similar to that which Ford uses (the Green Hills Software Integrated Development Environment). Ford also objects to a multitude of Plaintiffs' proposed tools on the grounds that they pose "a significant safety or security threat to Ford," they require the source code in write-access format in order to run, they "implicate[] the production by Ford of source code for systems far beyond the ETC system," or they are duplicative of other tools that Ford has agreed Plaintiffs may use. Ford disagrees with Plaintiffs that many of their listed tools may be used with source code in a read-only format. Importantly, Ford asserts that its simulators require source code to be in write-access format; however, Ford's compilers, assemblers, linkers, and debuggers do not require write-access format. Ultimately, Ford has agreed to the use of six tools or types of tools proposed by Plaintiffs, including the static analysis tools used by Ford, Windows PowerShell version 5 (a script developing tool), Veracrypt (an encryption tool), PC-Lint (a static analysis tool), RSM (a metrics tool), and Beyond Compare (a file comparison tool).
Federal Rule of Civil Procedure 26(b)(1) provides that:
While the claims and defenses raised in the case should be the focus of discovery, broader discovery is permitted when justified by the particular needs of the case. Fed. R. Civ. P. 26(b)(1), advisory committee notes (2000). In general, information is relevant, and thus discoverable, if it `"bears on, or . . . reasonably could lead to other matter[s] that could bear on, any issue that is or may be in the case. Although `the pleadings are the starting point from which relevancy and discovery are determined . . . [r]elevancy is not limited by the exact issues identified in the pleadings, the merits of the case, or the admissibility of discovered information.'" Kidwiler v. Progressive Paloverde Ins. Co., 192 F.R.D. 193, 199 (N.D.W.Va. 2000) (internal citations omitted). In many cases, "the general subject matter of the litigation governs the scope of relevant information for discovery purposes." Id. The party resisting discovery, not the party seeking discovery, bears the burden of persuasion. See Kinetic Concepts, Inc. v. ConvaTec Inc., 268 F.R.D. 226, 243-44 (M.D.N.C. 2010)(citing Wagner v. St. Paul Fire & Marine Ins. Co., 238 F.R.D. 418, 424-25 (N.D.W.Va. 2006)).
Simply because information is discoverable under Rule 26, however, "does not mean that discovery must be had." Schaaf v. SmithKline Beecham Corp., 233 F.R.D. 451, 453 (E.D.N.C. 2005) (citing Nicholas v. Wyndham Int'l, Inc., 373 F.3d 537, 543 (4th Cir. 2004)). For good cause shown under Rule 26(c), the court may restrict or prohibit discovery when necessary to protect a person or party from annoyance, embarrassment, oppression, or undue burden or expense. Fed. R. Civ. P. 26(c). In addition, Rule 26(b)(2)(C) requires the court, on motion or on its own, to limit the frequency and extent of discovery, when (1) "the discovery sought is unreasonably cumulative or duplicative;" (2) the discovery "can be obtained from some other source that is more convenient, less burdensome, or less expensive;" (3) "the party seeking the discovery has already had ample opportunity to collect the requested information by discovery in the action;" or (4) "the burden or expense of the proposed discovery outweighs its likely benefit, considering the needs of the case, the amount in controversy, the parties' resources, the importance of the issues at stake in the action, and the importance of the discovery in resolving the issues." Fed. R. Civ. P. 26(b)(2)(C)(i)-(iii). This rule "`cautions that all permissible discovery must be measured against the yardstick of proportionality.'" Lynn v. Monarch Recovery Mgmt., Inc., 285 F.R.D. 350, 355 (D.Md. 2012) (quoting Victor Stanley, Inc. v. Creative Pipe, Inc., 269 F.R.D. 497, 523 (D.Md. 2010)). To insure that discovery is sufficient, yet reasonable, district courts have "substantial latitude to fashion protective orders." Seattle Times Co. v. Rhinehart, 467 U.S. 20, 36, 104 S.Ct. 2199, 81 L.Ed.2d 17 (1984).
Federal Rule of Civil Procedure 26(c)(1)(G) allows the court, for good cause, to issue an order "requiring that a trade secret or other confidential research, development, or commercial information not be revealed or be revealed only in a specified way." In order for the court to apply the rule, two criteria must exist. First, the material sought to be protected must be "a trade secret or other confidential research, development, or commercial information." Second, there must be a "good cause" basis for granting the restriction. The party seeking protection bears the burden of establishing both the confidentiality of the material and the harm associated with its disclosure. Deford v. Schmid Prods. Co., 120 F.R.D. 648, 653 (D.Md. 1987) (citing Cipollone v. Liggett Group, Inc., 785 F.2d 1108, 1121 (3d Cir. 1986)). Once these elements are demonstrated, the burden shifts to the party seeking disclosure to show that the material is relevant and necessary to its case. Empire of Carolina, Inc. v. Mackle, 108 F.R.D. 323, 326 (D.C. Fla. 1985). The court "must balance the requesting party's need for information against the injury that might result if uncontrolled disclosure is compelled." Pansy v. Borough of Stroudsburg, 23 F.3d 772, 787 (3d Cir. 1994) (quoting Arthur R. Miller, Confidentiality, Protective Orders, and Public Access to the Courts, 105 Harv. L. Rev. 427, 432-33 (1991)).
If the court determines that disclosure is required, the issue becomes whether the materials should be "revealed only in a specified way." Fed. R. Civ. P. 26(c)(1)(G). "Whether this disclosure will be limited depends on a judicial balancing of the harm to the party seeking protection (or third persons) and the importance of disclosure to the public." Id. Factors to consider when deciding if and how to limit disclosure include:
Pansy, 23 F.3d at 787-91. The court exercises broad discretion in deciding "`when a protective order is appropriate and what degree of protection is required.'" Furlow v. United States, 55 F.Supp.2d 360, 366 (D.Md. 1999) (quoting Seattle Times Co., 467 U.S. at 36).
As discussed above, the parties disagree over the format in which the ETC source code must be produced. Beyond that, the parties also differ on what software tools Plaintiffs should be permitted to use in viewing and analyzing the ETC source code. Having considered the parties' submissions and the testimony adduced at the hearing, the Court finds that Plaintiffs are entitled to a read-only version of the ETC source code because the benefits that Plaintiffs seek from gaining access to the source code can be realized through a read-only format. As Dr. Kelly testified, most of what Mr. Jones demonstrated at the hearing can be performed on source code in a read-only format. (ECF No. 531 at 99). The only possible limitations of a read-only format are that the source code cannot be changed and faults cannot be injected directly into the source code by editing the code. (Id. at 89). However, as Ford and Dr. Kelly have pointed out, changing the source code would make the resulting code unrepresentative of the code contained in Plaintiffs' vehicles. (Id. at 32-33, 104). Moreover, Dr. Kelly asserted that fault injection can be performed in a way that does not implicate editing or adding lines to the source code. (Id. at 106). Under a proportionality analysis, at this juncture, any additional benefit of providing Plaintiffs with the code in a write-access format does not outweigh Ford's concerns about the security of its highly proprietary ETC source code.
As for the specific programs that Plaintiffs' experts may use, Ford informed the Court that its compiler and debugger could be used on read-only code, but objected to allowing Plaintiffs to use a compiler. The Court finds that Plaintiffs should be permitted to use a compiler in analyzing the source code as the compiler may alert Plaintiffs to certain errors in the code. Moreover, the debugger requires compiled code. (ECF No. 531 at 49, 55-56, 86, 100). The Court agrees with both parties that Plaintiffs should be required to use a compiler that is similar to the compiler that Ford used in developing the ETC source code, so that there is no discrepancy as to how the code was compiled when it was analyzed by Plaintiffs' experts.
Ford offers multiple arguments against the remainder of Plaintiffs' proposed tools. First, Ford argues that certain programs may allow modification of the ETC source code in RAM (random access memory) without being able to trace any edits, or permit Plaintiffs' experts to create additional tools for source code review without Ford's knowledge. However, Plaintiffs and their experts are instructed by this opinion that they are not to change or edit the ETC source code in any way when performing their analysis of the code. Furthermore, Plaintiffs shall be required to disclose all of the software tools that they plan to use in analyzing the source code
In addition, Ford contends that some of Plaintiffs' proposed tools will require the production of source code beyond the ETC source code. The Court need not address that concern at this time given that Plaintiffs have formally requested the ETC source code only; therefore, they have been granted access to the ETC source code only. If after reviewing the ETC source code, Plaintiffs believe that other portions of source code are indispensable to completing a proper and efficient analysis of the ETC source code as it is relevant to the claims and defenses in this litigation, then Plaintiffs may return to the Court and demonstrate good cause for the production of additional portions of source code.
Ford also objects to the use of certain proposed tools on the grounds that they are duplicative of other proposed tools, or that Ford does not use those tools in developing its source code. For example, Ford suggests that Plaintiffs' experts should not be permitted to use multiple static analysis tools. Yet, the NASA Engineering and Safety Center recognized the value of utilizing different static analysis tools in conducting its review of Toyota's source code:
NASA Engineering and Safety Center, National Highway Traffic Safety Administration Toyota Unintended Acceleration Investigation — Appendix A, 9 (2011), available at www.nhtsa.gov/staticfiles/nvs/pdf/NASA_FR_Appendix_A_Software.pdf.
In sum, Ford's objections to the read-only compatible tools proposed by Plaintiffs are unconvincing. The Court
Having concluded that Plaintiffs are entitled to the ETC source code in a read-only format and that Plaintiffs may use the listed tools that are compatible with such a format, the Court finds that Plaintiffs should start their analysis of the ETC source code with the vehicle models that they have already tested. As noted above, Plaintiffs' counsel has represented that testing was performed on two models and that Plaintiffs' experts were able to recreate sudden acceleration events during that testing. (ECF No. 531 at 72, 75). Plaintiffs' experts should begin their source code analysis on the source code used in those two models (the 2005 Explorer and the 2005 Mustang). (Id. at 76). After an initial review of the ETC source code for those models, Plaintiffs shall report back to the Court with an estimate of the time that it will take to complete the ETC source code analysis for those two models. Plaintiffs shall also provide the Court with an estimate of time that it will take to analyze the ETC source code for the additional models at issue in this litigation, as well as a proposed schedule setting forth in order of priority the vehicle models and years from which the ETC source code shall be analyzed.
Finally, the parties are
The Clerk is directed to provide a copy of this Order to counsel of record and any unrepresented party.