Opinion & Order

ANDREW L. CARTER, Jr., District Judge.

Before the Court are Plaintiffs' objections, filed pursuant to Rule 72(a) of the Federal Rules of Civil Procedure, to Magistrate Judge Andrew J. Peck's rulings rendered during a February 8, 2012 discovery conference and his February 24,2012 opinion and order (dkt. no. 96). Judge Peck's rulings and written order discussed, inter alia, the use of the predictive coding software, a computer assisted form of review. The Court assumes the parties' familiarity with the facts and the predictive coding method.

Plaintiffs and Defendant MSLGroup entered into a stipulation governing MSLGroup's production of electronically stored information ("ESI protocol"). The substance of the ESI protocol resulted from a series of court conferences and party discussions, but the February 8 conference set the final parameters for the protocol. Judge Peck so-ordered the ESI protocol on February 22,2012. (Dkt. No. 92.) The last paragraph of the protocol provides that Plaintiffs object to the entire ESI protocol, but signed the agreement because it reflects Judge Peck's discovery rulings and Judge Peck rejected Plaintiffs' version of the ESI protocol.

Plaintiffs object to the February 8 discovery rulings, the ESI protocol, and the February 24 opinion and order, arguing, inter alia, that the predictive coding method contemplated in the ESI protocol lacks generally accepted reliability standards, that the use of such method violates Fed. R. Civ. P. 26 and Federal Rules of Evidence 702, that Judge Peck improperly relied on outside documentary evidence in his February 24 opinion and order, that MSLGroup's expert is biased because the use of the predictive coding method will reap financial benefits for the company, that Judge Peck failed to hold an evidentiary hearing, and that he adopted MSLGroup's version of the ESI protocol on an insufficient record. Plaintiffs request that the Court overturn the Magistrate Judge's rulings because they are erroneous and contrary to law.1 Plaintiffs also submitted a letter requesting that Judge Peck recuse himself from the action, which Judge Peck denied on April 2, 2012, but allowed them to file a formal motion. Plaintiffs filed their recusal motion on April 13, 2012, incorporating similar arguments made in their Rule 72(a) objections. (Dkt. No. 169.)

Rule 72(a) provides that for nondispositive orders issued by a magistrate judge, "[t]he district judge in the case must consider timely objections and modify or set aside any part of the order that is clearly erroneous or is contrary to law." Fed. R. Civ. P. 72(a); see also 28 U.S.C. § 636(b)(1)(A). "Under this highly deferential standard of review, magistrates are afforded broad discretion in resolving [non-dispositive] disputes and reversal is appropriate only if their discretion is abused." AMBAC Fin. Servs., LLC v. Bay Area Toll Auth., No. 09 Civ. 7062, 2010 WL 4892678, at *2 (S.D.N.Y. Nov. 30, 2010) (citation omitted). A magistrate judge's ruling is considered "contrary to law" when it "fails to apply or misapplies relevant statutes, case law, or rules of procedure." In re Comverse Tech., Inc. Sec. Litig., No. 06 Civ. 1825, 2007 WL 680779, at *2 (E.D.N.Y. Mar. 2, 2007). "The reviewing court must be left with the definite and firm conviction that a mistake has been committed to overturn the magistrate judge's resolution of a nondispositive matter." AMBAC Fin. Servs., 2010 WL 4892678, at *2 (citation and internal quotation omitted). "Matters concerning discovery generally are considered `nondispositive' of the litigation." Thomas E. Hoar, Inc. v. Sara Lee Corp., 900 F.2d 522, 525 (2d Cir. 1990).

Mindful of this highly deferential standard of review, the Court adopts Judge Peck's rulings because they are well reasoned and they consider the potential advantages and pitfalls of the predictive coding software. The Court has thoroughly reviewed the ESI protocol along with the parties' submissions.2 At the outset, the Court notes that Plaintiffs and Judge Peck disagree about the scope of Plaintiffs' acquiescence concerning the use of the method. Judge Peck's written order states that Plaintiffs have consented to its use, (Opinion and Order at 17 ("The decision to allow computer-assisted review in this case was relatively easy — the parties agreed to its use (although disagreed about how best to implement such review."))), while Plaintiffs argue that Judge Peck's order mischaracterizes their position (Pl. Reply, dated March 19, 2012, at 4-5). Nevertheless, the confusion is immaterial because the ESI protocol contains standards for measuring the reliability of the process and the protocol builds in levels of participation by Plaintiffs. It provides that the search methods will be carefully crafted and tested for quality assurance, with Plaintiffs participating in their implementation. For example, Plaintiffs' counsel may provide keywords and review the documents and the issue coding before the production is made. If there is a concern with the relevance of the culled documents, the parties may raise the issue before Judge Peck before the final production. Further, upon the receipt of the production, if Plaintiffs determine that they are missing relevant documents, they may revisit the issue of whether the software is the best method. At this stage, there is insufficient evidence to conclude that the use of the predictive coding software will deny Plaintiffs access to liberal discovery.

Plaintiffs' arguments concerning the reliability of the method are also premature. It is difficult to ascertain that the predictive software is less reliable than the traditional keyword search. Experts were present during the February 8 conference and Judge Peck heard from these experts. The lack of a formal evidentiary hearing at the conference is a minor issue because if the method appears unreliable as the litigation continues and the parties continue to dispute its effectiveness, the Magistrate Judge may then conduct an evidentiary hearing. Judge Peck is in the best position to determine when and if an evidentiary hearing is required and the exercise of his discretion is not contrary to law. Judge Peck has ruled that if the predictive coding software is flawed or if Plaintiffs are not receiving the types of documents that should be produced, the parties are allowed to reconsider their methods and raise their concerns with the Magistrate Judge. The Court understands that the majority of documentary evidence has to be produced by MSLGroup and that Plaintiffs do not have many documents of their own. If the method provided in the protocol does not work or if the sample size is indeed too small to properly apply the technology, the Court will not preclude Plaintiffs from receiving relevant information, but to call the method unreliable at this stage is speculative.3

There simply is no review tool that guarantees perfection. The parties and Judge Peck have acknowledged that there are risks inherent in any method of reviewing electronic documents. Manual review with keyword searches is costly, though appropriate in certain situations. However, even if all parties here were willing to entertain the notion of manually reviewing the documents, such review is prone to human error and marred with inconsistencies from the various attorneys' determination of whether a document is responsive. Judge Peck concluded that under the circumstances of this particular case, the use of the predictive coding software as specified in the ESI protocol is more appropriate than keyword searching. The Court does not find a basis to hold that his conclusion is clearly erroneous or contrary to law. Thus, Judge Peck's orders are adopted and Plaintiffs' objections are denied.

The Court also concludes that it should not reject Judge Peck's rulings concerning the production of W-2s. Judge Peck previously ruled that Plaintiffs may visit defense counsel's office and identify the W-2s they want. He also quashed Plaintiffs' third-party subpoena on Automatic Data Processing Services. Plaintiffs and MSLGroup dispute whether the W-2s that Plaintiffs identified have actually been produced. (Compare PI. Reply at 10 with Def. Opp. at 25.) To the extent that Plaintiffs are missing the W-2s that they identified to MSLGroup, the Court instructs them to send MSLGroup an email noting the missing W-2s within two days of this order. MSLGroup should produce those specified W-2s within three business days of Plaintiffs' email. If there are additional disputes about the missing W-2s, they should return to Judge Peck. Further, Judge Peck's decision to delay production of emails from MSLGroup's chief executive officer until "Phase II" of the ESI protocol is not erroneous or contrary to law. Judge Peck did not make a final determination about whether MSLGroup should produce those emails. The Court reminds the parties that it affords Judge Peck's non-dispositive rulings great deference and that magistrate judges generally have broad latitude with respect to discovery issues.

SO ORDERED.