ORDER & OPINION OF THE SPECIAL MASTER

DENNIS M. CAVANAUGH, Special Master.

This matter comes before the Special Master upon letter briefing submitted by Plaintiffs and Defendants (Daimler AG, Mercedes-Benz USA, LLC, and Robert Bosch LLC) related to the parties' discovery dispute involving the methodology Defendants should use to identify responsive documents and the provisions of a Search Term Protocol. After considering the submissions of the parties, based upon the following, it is the opinion of the Special Master that Defendants will not be compelled to utilize technology assisted review at this time, and that the Search Term Protocol as modified by the Special Master is adopted.

DISCUSSION

I. Methodology to Identify Responsive Documents

The parties disagree as to the methodology Defendants should use to identify responsive documents. Plaintiffs propose that the Defendants use predictive coding or technology assisted review ("TAR"), in which human reviewers and a computer engage in an interactive process to "train" the computer how to identify responsive documents based on properties and characteristics beyond simple search terms. Plaintiffs assert that this type of computer-assisted coding process yields significantly better results than either traditional human "eyes on" review of the full data set or the use of search terms. Plaintiffs argue that if the Court declines to compel Defendants to adopt TAR, the Court should enter its proposed Search Term Protocol.

Defendants argue that there is no authority for imposing TAR on an objecting party. Defendants further argue that this case presents a number of unique issues that would make developing an appropriate and effective seed set challenging, such as language and translation issues, unique acronyms and identifiers, redacted documents, and technical documents. Defendants thus contend they should be permitted to utilize their preferred custodian-and-search term approach.

While "the case law has developed to the point that it is now black letter law that where the producing party wants to utilize TAR for document review, courts will permit it" Rio Tinto PLC v. Vale S.A., 306 F.R.D. 125, 127 (S.D.N.Y. 2015), no court has ordered a party to engage in TAR over the objection of that party. The few courts that have considered this issue have all declined to compel predictive coding. See, e.g., City of Rockford v. Mallinckrodt ARD Inc., 326 F.R.D. 489, 493, 101 Fed. R. Serv. 3d 622 (N.D. Ill. 2018) (discussing the advantages of TAR but deferring to the parties' choice to use search terms); Hyles v. New York City, 2016 WL 4077114, *3 (S.D.N.Y. 2016) (refusing to order a party to use TAR and stating that a party is free to decide how to search so long as its process is reasonable); Rio Tinto PLC v. Vale S.A., 306 F.R.D. 125, 127 & n.2 (S.D. N.Y. 2015).

Despite the fact that it is widely recognized that "TAR is cheaper, more efficient and superior to keyword searching" Hyles v. New York City, No. 10-CIV-3119, 2016 WL 4077114, at *2 (S.D.N.Y. Aug. 1, 2016), courts also recognize that responding parties are best situated to evaluate the procedures, methodologies, and technologies appropriate for producing their own electronically stored information. Ibid. at *3 (citing The Sedona Principles: Second Edition, Best Practices Recommendations & Principles for Addressing Electronic Document Production, Principle 6 (available at www.TheSedonaConference.org)).

Here, Defendants object to the use of TAR, instead indicating they prefer to use the custodian-and-search term approach, which they assert is fair, efficient, and well-established. While the Special Master believes TAR would likely be a more cost effective and efficient methodology for identifying responsive documents, Defendants may evaluate and decide for themselves the appropriate technology for producing their ESI. Therefore, the Special Master will not order Defendants to utilize TAR at this time. However, Defendants are cautioned that the Special Master will not look favorably on any future arguments related to burden of discovery requests, specifically cost and proportionality, when Defendants have chosen to utilize the custodian-and-search term approach despite wide acceptance that TAR is cheaper, more efficient and superior to keyword searching. Additionally, the denial of Plaintiffs' request to compel Defendants to utilize TAR is without prejudice to revisiting this issue if Plaintiffs contend that Defendants' actual production is deficient.

II. Proposed Search Term Protocols

With respect to a proposed Search Term Protocol, the parties have conducted several meet and confers in an effort to come to an agreement. However, despite their best efforts, three provisions remain in dispute: validation; known responsive materials; and production of discrete collections.

A. Validation

Defendants explain that they have agreed to three different kinds of validation, including validation of documents that did not hit on search terms and a four-step post-production validation process. However, Defendants assert that Plaintiffs' proposal simply requires Plaintiffs to perform appropriate sampling and quality control, without providing any detail on how that sampling or quality control will be done. Moreover, Defendants take issue with the fact that Plaintiffs' proposal states that "Plaintiffs will not be obligated to collect or sample ESI that does not contain a search term." Defendants assert that in an effort to resolve the dispute, they have offered that the validation terms need not be completely reciprocal, so long as they contain some concrete and meaningful obligations that are transparent. Defendants explain that their proposal provides for the parties to meet and confer, at Defendants' request, to discuss the application of validation procedures based on the procedures identified in paragraph 12(a) of the protocol. Defendants believe their proposal takes into account interests such as practicability, reasonableness, and proportionality while also addressing the interests of providing transparency about the validation procedures that Plaintiffs will use and ensuring that Plaintiffs' collection and validation procedures are adequate.

Plaintiffs argue that while they will of course validate their searches, the same provisions that apply to Defendants cannot feasibly apply to Plaintiffs given the highly personal nature of Plaintiffs' email collections and the relatively small size of those collections. Plaintiffs argue that they have committed to meet and confer with Defendants to discuss each individual Plaintiff's search, and Plaintiffs will perform appropriate sampling and quality control to achieve an appropriate level of validation of Plaintiffs' search terms. Plaintiffs allege that Defendants are insisting on reciprocity through the "back door" by requiring Plaintiffs to continue discussing the issue and reserving their right to later have the Court order it.

The parties have engaged in protracted discussions regarding implementation of a Search Term Protocol and the Special Master applauds their sincere efforts to substantially narrow their disagreements. As the parties well know, collection, review, and production of ESI presents special challenges and requires "cooperation between opposing counsel and transparency in all aspects of preservation and production of ESI." William A. Gross Const. Assocs., Inc. v. Am. Mfrs. Mut. Ins. Co., 256 F.R.D. 134, 136 (S.D.N.Y. 2009).

With respect to the appropriate level of validation of Plaintiffs' search terms, after considering the arguments of the parties, the Special Master adopts Defendants' proposal as modified. The parties agree and case law dictates that appropriate validation be utilized to test search results. See e.g. In re Seroquel Products Liability Litig., 244 F.R.D. 650, 662 (M.D. Fla. 2007) (Baker, M.J.) ("[W]hile key word searching is a recognized method to winnow relevant documents from large repositories, use of this technique must be a cooperative and informed process.... Common sense dictates that sampling and other quality assurance techniques must be employed to meet requirements of completeness.").

While the parties have been able to reach agreement on the terms of Defendants' validation process, the parties are at an impasse regarding the level of validation of Plaintiffs' search term results. Plaintiffs explain that the discoverable ESI that an individual Plaintiff possesses will likely be contained in his or her email, and be intermingled with wholly irrelevant and personal private information. The Special Master agrees that the validation process described in section 12(a) may not be practicable to Plaintiffs as Plaintiffs are individuals with vastly smaller amounts of discoverable ESI than Defendants, who are large corporations. Defendants also appear to agree that the validation terms need not be completely reciprocal. However, Plaintiffs' proposal does not articulate how it will perform appropriate sampling and quality control measures to achieve the appropriate level of validation. As no articulable alternative process has been proposed by Plaintiffs, the Special Master will adopt Defendants' protocol to the extent that it will require the parties, at Defendants' request, to meet and confer concerning the application of validation procedures described in paragraph 12(a) to Plaintiffs, if the parties are unable to agree to a procedure. The Special Master encourages the parties to work together to develop a reasonable procedure for the validation of Plaintiffs' search terms, however, should the parties need to, they may seek a further order from the Court that will explicitly dictate which, if any, provisions described in paragraph 12(a) must be utilized by Plaintiffs in validating their search terms.

B. Known Responsive Documents & Discrete Collections

With respect to the parties' disagreement concerning known responsive materials, Defendants argue that Plaintiffs' protocol would require the production of all documents and ESI "known" to be responsive, regardless of how or by whom the materials are "known" to be responsive. Defendants assert that the proposal is vague, exceedingly burdensome, and provides no clear standard for the court to administer or the parties to apply. Defendants argue that their proposal provides that documents or ESI known to be responsive will not be withheld on the basis that such documents or ESI were not responsive to search methodology described in the protocol. Defendants argue that their proposal is proportional to the needs of the case, and addresses Plaintiffs' underlying concern that the use of search terms may not capture responsive documents or ESI and should not be used to withhold them.

Similarly, Defendants argue that Plaintiffs' request for "folders or collections of information that are known to contain documents likely to be responsive to a discovery request" is overly broad and flouts the requirement that discovery be proportional to the needs of the case. Defendants argue that their network drive folders can contain terabytes of data which would be burdensome to collect and produce. Defendants assert that their compromise proposal accounts for proportionality: it provides for the production, without applying search terms, of discrete folders or collections of information of a reasonable volume that are known to predominantly contain documents likely to be responsive to discovery requests, and it allows the parties to meet and confer if the receiving party identifies any deficiency in the production. Defendants point to Gamboa, et al., v. Ford Motor Co., et al., Civil Action No. 18-10106 (E.D. Mich. Filed Dec. 16, 2019), ECF No. 151 at 7, wherein the parties were required to meet and confer if a discrete document folder or collection was too big to make review of each document proportional to the needs of the case.

Plaintiffs argue that the Court has already ruled that the parties must produce without the use of search terms ESI known to be responsive to discovery requests in this case. Plaintiffs argue that Defendants' edits water-down provisions requiring the parties to produce documents or ESI known by the producing party to be responsive to a discovery request. Similarly, Plaintiffs argue that Defendants' edits to additional paragraphs are intended to dodge the parties' obligation to produce without the use of search terms "discrete folders or collections of information that are known to contain documents likely to be responsive to a discovery request." Plaintiffs argue that given the shortcomings of the search terms methodology, there is no reason why the parties should not produce ESI collections known to be responsive to a discovery request.

During the November 17, 2019 status conference, Plaintiffs raised the issue of Defendants' obligation to produce responsive materials that are known to be responsive. Defendants indicated that they agreed to produce materials that are known to be responsive. The parties are thus in agreement that Defendants will produce known materials responsive to a discovery request and will not withhold known relevant documents on the basis that they were not responsive to any search methodology described in the Search Term Protocol. The parties' disagreement comes down to the appropriate wording of the Search Term Protocol.

In light of Defendants' concern, the Special Master will modify the Search Term Protocol to require production of materials that are "reasonably known" to be responsive. The Special Master notes that Defendants already agreed to produce materials that are known to be responsive at the November status conference. Nevertheless, the Special Master will modify the Search Term Protocol to require production of materials that are "reasonably known" to be responsive. In affording this modification, the Special Master intends to provide Defendants some flexibility should they inadvertedly fail to produce documents, which Plaintiffs believe were known to be responsive. If during discovery the parties develop an issue with respect to the production or lack of production of reasonably known responsive documents, the parties may seek the appropriate remedy and order from the Court.

Finally, with respect to folders or collections of information, the Special Master will require the parties to collect this material to the extent it is reasonably known to the producing party. To the extent the folder or collections contain an extensive volume of material, the Special Master will require the parties to meet and confer if a party believes a discrete document folder or collection of information that is relevant to a claim or defense is too voluminous to make review of each document proportional to the needs of the case. Should the parties be unable to resolve their dispute during the meet and confer process, they may seek a further order from the Court.