MEMORANDUM AND ORDER

JAMES C. FRANCIS IV, United States Magistrate Judge.

This is an employment discrimination action in which the plaintiffs allege that their former employers, defendants Goldman, Sachs & Co. and The Goldman Sachs Group, Inc. (collectively, "Goldman Sachs" or the "defendants"), denied them equal compensation and opportunity for promotion in violation of Title VII of the Civil Rights Act of 1964, 42 U.S.C. § 2000e et seq., and New York City Human Rights Law, N.Y.C. Admin. Code § 8-101 et seq. The plaintiffs have moved for certification of a class of similarly situated employees, and the parties have submitted expert evidence in connection with the motion. Each side now moves to exclude part of the other's expert testimony on the basis that it is inadmissible pursuant to Rule 702 of the Federal Rules of Evidence and the principles set forth in Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 597, 113 S.Ct. 2786, 125 L.Ed.2d 469 (1993). After discussing the relevant legal principles, I will address each challenged expert in turn.1

Legal Framework

A. Applicability of Daubert to Class Certification

Neither the Supreme Court nor the Second Circuit has definitively decided whether the Daubert standard governs the admissibility of expert evidence submitted at the class certification stage. See Wal-Mart Stores, Inc. v. Dukes, 564 U.S. 338, ___, 131 S.Ct. 2541, 2553-54, 180 L.Ed.2d 374 (2011); In re U.S. Foodservice Inc. Pricing Litigation, 729 F.3d 108, 129 (2d Cir.2013). Most courts that have addressed the issue have concluded that the Daubert test does apply in this context. See Sher v. Raytheon Co., 419 Fed.Appx. 887, 890-91 (11th Cir.2011); American Honda Motor Co. v. Allen, 600 F.3d 813, 815-16 (7th Cir.2010) ("We hold that when an expert's report or testimony is critical to class certification, as it is here, ... a district court must conclusively rule on any challenge to the expert's qualifications or submissions prior to ruling on a class certification motion."); see also Ge Dandong v. Pinnacle Performance Ltd., No. 10 Civ. 8086, 2013 WL 5658790, at *13 (S.D.N.Y. Oct. 17, 2013). Several rationales support this view. First, Danbert is an amplification of Rule 702, and the Federal Rules of Evidence apply generally to "proceedings" in the courts of the United States, with certain exceptions not applicable here. Fed.R.Evid. 101, 1101. Second, the Supreme Court has directed district courts to conduct a "rigorous analysis" in class certification proceedings to determine whether the requirements of Rule 23 of the Federal Rules of Civil Procedure are met, General Telephone Co. of Southwest v. Falcon, 457 U.S. 147, 161, 102 S.Ct. 2364, 72 L.Ed.2d 740 (1982), and relaxed evidentiary standards would be inconsistent with this prescription. Finally, the Supreme Court itself has expressed skepticism at the suggestion that Daubed would not apply to expert testimony at the class certification stage. Dukes, 564 U.S. at ___, 131 S.Ct. at 2553-54; see U.S. Foodservice, 729 F.3d at 129 (characterizing Dukes as "suggesting that a Daubert analysis may be required at least in some circumstances"). While the Supreme Court's observation was plainly dicta, a district court is well-advised to heed it. See Tiffany (NJ) Inc. v. eBay Inc., 600 F.3d 93, 108 (2d Cir.2010) (finding Supreme Court dicta to be persuasive authority).

Courts that do not find Danbert applicable to class certification generally rely on two arguments. First, they note that a class determination motion is "inherently tentative" and subject to later revision. See In re Zurn Pex Plumbing Products Liability Litigation, 644 F.3d 604, 613 (8th Cir.2011) (citing Coopers & Lybrand v. Livesay, 437 U.S. 463, 469 n. 11, 98 S.Ct. 2454, 57 L.Ed.2d 351 (1978)). But this rationale must yield to the mandate to conduct a rigorous analysis prior to certifying a class. Second, some courts reason that the Daubert standard need not be enforced at the time of class certification because, at that stage, the expert opinion is being submitted to a judge and not to a jury. See id. Certainly, a court may delay a Daubert determination in order to consider all the testimony proffered, but that does not mean that a court can rely on inadmissible evidence in making its ultimate determination.

The cases that hold Daubert to be applicable at the class certification stage, then, are more persuasive. However, the scope of the Daubert analysis is cabined by its purpose at this stage: "the inquiry is limited to whether or not the expert reports are admissible to establish the requirements of Rule 23." Ge Dandong, 2013 WL 5658790, at *13 (brackets and internal quotation marks omitted). It would be premature to decide whether aspects of an expert opinion that go exclusively to the merits and not to the elements of Rule 23 would be admissible in subsequent proceedings.

B. Legal Standard

Rule 702 of the Federal Rules of Evidence provides:

Daubert reaffirmed that "the trial judge [has] the task of ensuring that an expert's testimony both rests on a reliable foundation and is relevant to the task at hand." Daubert, 509 U.S. at 597, 113 S.Ct. 2786. Performing its function as "gatekeeper" of expert testimony requires the court to conduct a "rigorous examination of the facts on which the expert relies, the method by which the expert draws an opinion from those facts, and how the expert applies the facts and methods to the case at hand." Amorgianos v. National Railroad Passenger Corp., 303 F.3d 256, 267 (2d Cir.2002). This filtering role applies "not only to testimony based on `scientific' knowledge, but also to testimony based on `technical' and `other specialized' knowledge." Kumho Tire Co. v. Carmichael, 526 U.S. 137, 141, 119 S.Ct. 1167, 143 L.Ed.2d 238 (1999). However, in such cases "the reliability inquiry may instead focus upon personal knowledge and experience of the expert.'" Davis v. Carroll, 937 F.Supp.2d 390, 412 (S.D.N.Y.2013) (quoting Bah v. Nordson Corp., No. 00 Civ. 9060, 2005 WL 1813023, at *7 (S.D.N.Y. Aug. 1, 2005)).

The proponent of expert testimony must establish its reliability and admissibility under Rule 702 by a preponderance of the evidence. United States v. Williams, 506 F.3d 151, 160 (2d Cir.2007). There is a presumption that expert evidence is admissible, Borawick v. Shay, 68 F.3d 597, 610 (2d Cir.1995), and "the rejection of expert testimony is the exception rather than the rule." Fed.R.Evid. 702 advisory committee's note (2000); accord Floyd v. City of New York, 861 F.Supp.2d 274, 287 (S.D.N.Y.2012). However, "when an expert opinion is based on data, a methodology, or studies that are simply inadequate to support the conclusions reached, Dambert and Rule 702 mandate the exclusion of that unreliable opinion testimony." Amorgianos, 303 F.3d at 266.

The Challenged Experts

A. Dr. Henry S. Farber

Goldman Sachs moves to exclude the report of Dr. Henry S. Farber, a statistician who analyzed the impact of gender on compensation and promotion at the firm. Dr. Farber is the Hughes-Rogers Professor of Economics at Princeton University, where he has been on the faculty since 1991 and teaches courses in labor economics and econometrics. (Expert Report of Henry S. Farber in Connection with Chen-Oster v. Goldman Sachs dated Feb. 17, 2014 ("Farber Report"), attached as Exh. 8 to Declaration of Barbara B. Brown dated June 13, 2014 ("Brown Decl."), ¶ 2). Dr. Farber received a B.S. degree in economics from Rensselaer Polytechnic Institute in 1972, an M.S. in industrial and labor relations from Cornell University in 1974, and a Ph.D. from Princeton in 1977, after which he taught economics at the Massachusetts Institute of Technology before joining the faculty at Princeton. (Farber Report, ¶ 2).

Dr. Farber utilized multiple regression analysis as his primary statistical tool in studying Goldman Sachs' work force. (Farber Report, ¶ 36). He first created a model in which he sought to predict the compensation of employees based on a number of factors, which included the year, the division in which the employee worked,2 geographic office, education, prior related work experience, experience at Goldman Sachs,3 lateral or direct hiring status, and job group. (Farber Report, ¶¶ 44-46). When the regression is performed, a coefficient is derived for each factor, which represents the relative impact of that factor on compensation. (Farber Report, ¶ 36). However, there is substantial variation in pay that is not explained by the factors included in the model, and the deviation between predicted and actual compensation for any individual is identified as the residual. (Farber Report ¶¶ 37-38). To the extent that the residuals are systematically related to the employees' genders and indicate that women are paid less than men even after taking into consideration the other factors included in the model, discrimination may be inferred. (Farber Report, ¶ 39). Dr. Farber tested how likely it was that the residual differences in compensation were due to random variation or other unmeasurable factors, rather than to discrimination, by including gender as a variable in his model and then ascertaining whether being a woman had a statistically significant impact. (Farber Report, ¶¶ 40-43).

Based on this methodology, Dr. Farber found that female Associates at Goldman Sachs earned 7.6 percent less than male Associates after accounting for the factors in his model. (Farber Report, ¶ 55 & Table 6). The t-statistic for this figure is 5.10, which exceeds the generally accepted threshold for statistical significance of 1.96.4 (Farber Report, ¶ 43 & Table 6). Similarly, Dr. Farber determined that female Vice Presidents were paid 21.36 percent less than similarly-situated male Vice Presidents, and that this figure was also statistically significant, with a t-statistic of 9.88. (Farber Report, ¶ 56 & Table 7).

Dr. Farber also studied the relationship between gender and promotion from Vice President to Managing Director. The analysis was much the same as in his study of compensation, in that he controlled for the same predictive factors. (Farber Report, ¶ 84). However, because the question of whether someone is promoted or not has a binary answer, he used what is known as a probit model. (Farber Report, ¶ 84). First, he created a benchmark model of promotion for male Vice Presidents from which he could project the probability of promotion for similarly-situated females in the absence of discrimination. (Farber Report, ¶¶ 86-87). He found that while the promotion rate for men was 5.33 percent and the expected promotion rate for women was 5.12 percent after controlling for the predictive factors, the actual promotion rate for women was 3.96 percent. (Farber Report, Table 20). In absolute terms, his benchmark model showed that while 82 women would be expected to receive promotions, only 63 did, a shortfall of 19 promotions. (Farber Report, ¶ 89 & Table 20). These results were statistically significant, with a t-statistic of 2.59. (Farber Report, ¶ 89).

Finally, Dr. Farber analyzed the relationship between gender and the two primary performance measures utilized by Goldman Sachs, as well as the relationship between those measures and compensation. As discussed in greater detail in connection with the motion for class certification, Goldman Sachs evaluates its professionals using programs known as "360 review" and "quartiling." The former involves the derivation of a performance score from interviews with a range of managers, peers, and internal clients. The latter is a policy by which a supervisor is required to classify an established percentage of subordinates in one of five categories based on performance.

Dr. Farber again utilized his regression model and found that, after controlling for the predictive factors, women Associates and Vice Presidents were scored lower on the 360 review than their male counterparts and that these results were statistically significant. (Farber Report, ¶¶ 67-70 & Tables 14, 15).5 Similarly, using a probit model to study quartiling, Dr. Farber determined that, after taking into account the predictive factors, female Associates were 7.1 percentage points less likely to be assigned to the top quartile than male Associates, that female Vice Presidents were 4.5 percentage points less likely to be assigned to the top quartile than male Vice Presidents, and that these figures were statistically significant. (Farber Report, ¶¶ 60, 63-64 & Tables 10, 11).

Finally, Dr. Farber analyzed the impact of 360 review scores, quartile placement, and a combination of the two on compensation.6 In each case, the addition of the performance variables reduced but did not eliminate the pay gap ascribed to gender. When both the 360 review score and quartile were taken into account, the differential in compensation between female Associates and male Associates declined from 7.7 percent to 3.3 percent. (Farber Report, ¶ 77 & Table 16). Similarly, when those factors were included in the analysis for Vice Presidents, the pay differential declined from 21.8 percent to 16.9 percent. (Farber Report, ¶ 79 & Table 17). The gender pay gaps remained statistically significant, however, even after including the performance measures in the model. (Farber Report, ¶¶ 77, 79 & Tables 16, 17). Thus, Dr. Farber concluded that the 360 review and quartiling tools contribute to gender differences in compensation at Goldman Sachs. (Farber Report, ¶ 82).

The defendants' challenges to Dr. Farber's report can be grouped into three broad arguments: (1) he has failed to include important measures of performance and productivity that would potentially explain the differentials that he instead attributes to gender discrimination; (2) he does not adequately account for job function; and (3) his model improperly aggregates data from disparate business units. I will address each contention in turn.

1. Performance & Productivity

There are two straightforward answers to the complaint that Dr. Farber failed to include in his model two measures of performance — 360 review and quartile placement — that might be considered predictors of compensation and promotion. First, Dr. Farber reasonably suspected that these variables were tainted, that is, that they themselves are affected by gender bias. (Farber Report, ¶ 58). Second, he did, in fact, incorporate these factors into one iteration of his analysis, and doing so did not substantively alter his conclusions.

"Regression analyses in discrimination cases attempt to control for the legitimate reasons for pay disparities through the use of explanatory variables. But, illegitimate reasons — reasons themselves representative of the unlawful discrimination at issue — should be excluded from the regression (or otherwise dealt with) to avoid underestimating the significance of a disparity." Morgan v. United Parcel Service of America, Inc., 380 F.3d 459, 469-70 (8th Cir.2004); see Smith v. Xerox Corp., 196 F.3d 358, 371 n. 11 (2d Cir.1999), overruled on other grounds by Meacham, v. Knolls Atomic Power Laboratory, 461 F.3d 134 (2d Cir.2006); Ellis v. Costco Wholesale Corp., 285 F.R.D. 492, 524 (N.D.Cal.2012); McReynolds v. Sodexho Marriott Services, Inc., 349 F.Supp.2d 1, 26 (D.D.C.2004); D. James Greiner, Causal Inference in Civil Rights Litigation, 122 Hare. L.Rev. 533, 546-49 (2008). Of course, merely assuming that a variable is tainted is not a sufficient basis for omitting it from a regression model if it would otherwise add explanatory power. See Morgan, 380 F.3d at 470-71; Valentino v. U.S. Postal Service, 511 F.Supp. 917, 944 (D.D.C.1981). But here, Dr. Farber demonstrated that women were graded lower in both the 360 review and the quartiling process even after controlling for measurable and consistently available predictors of performance. Thus, there was a legitimate basis for him to omit these performance measures from his analysis on the ground that they were themselves subject to bias and would therefore mask discrimination.

More to the point, notwithstanding his qualms, Dr. Farber conducted a study in which he did incorporate Goldman Sachs' performance measures in his model, both separately and in tandem. As discussed above, although the analyses that control for the performance variables result in a less dramatic disparity in compensation between men and women, a statistically significant pay differential remains.

The defendants also argue that Dr. Farber should have incorporated productivity measures such as number of trades, sales credits, or deal fees in his model, since these are critical factors in determining compensation and promotion. (Defendants' Reply Memorandum in Support of Their Motion to Strike Reports, Opinion, and Testimony of Plaintiffs' Expert Dr. Henry S. Farber ("Def. Farber Reply") at 7-8). But there are practical barriers to this approach. First, these types of measures only apply to one-third of Goldman Sachs' professional employees (Def. Reply at 8), so there would be concerns about the completeness of the data to an even greater degree than was the case with respect to the performance metrics. Second, these measures are not likely to be comparable to one another; for example, number of trades is measured on a wholly different scale than the value realized from a deal. Third, these measures would introduce a problem of multicollinearity. "In statistical analysis, multi-colinearity [sic] is a correlation between two independent variables that is so strong that it is impossible statistically to separate them out and tell which of those two effects the dependent variable; which of those two is the cause." United States v. Charleston County, 316 F.Supp.2d 268, 300 n. 36 (D.S.C.2003) (internal quotation marks and citation omitted); see Gartenberg v. Merrill Lynch Asset Management, Inc., 573 F.Supp. 1293, 1313 n. 19 (S.D.N.Y.1983); Fisher, supra at 713 ("[T]he less multicollinearity is present, the better able one will be to separate out the effects of interest."). If the defendants are correct and productivity is central to how employees are evaluated, then this factor would likely be collinear with the 360 review and quartiling measures, which Dr. Farber has accounted for.

This last point is more than simply a technical statistical issue. In the 360 review, professionals are supposed to be evaluated on the basis of overall commercial effectiveness and professional performance, among other factors. (Deposition of Jessica Kung ("Kung Dep."), attached as Exhs. 9 and 9A to Declaration of Theodore O. Rogers, Jr., dated July 3, 2014 ("Rogers Decl."), at 305-07). Moreover, the functional definition of these traits varies depending upon the types of skills most relevant to a particular business unit. (Deposition of Caroline Heller Sberloti ("Heller Dep."), attached as Exhs. 16 and 16A to Rogers Decl., at 271-72; Deposition of Bruce Larson, attached as Exh. 11 to Rogers Decl., at 169-70). Similarly, quartiling designations are based on the employee's relative "performance, contribution and potential," including "[1]ong term commercial impact." (Guidelines for Manager Performance Rank, attached as part of Exh. 10C to Rogers Decl., at 1; see also Manager Performance Rating Toolkit, attached as Exh. 10B to Rogers Decl., at 3-4). Thus, the productivity that Goldman Sachs considers a crucial variable is already part of Dr. Farber's model in the form of the performance measures.

2. Job Function

One of the factors that Dr. Farber included in his model was the affirmative action job group classification, referred to as the AAP code. The defendants argue that the AAP code is irrelevant because it is not used to make evaluation, compensation, or promotion decisions and because 70 percent of the professionals in the putative class fall into one of three AAP job codes. (Def. Farber Reply at 9-10). According to Goldman Sachs, Dr. Farber should have used more granular job functions as a variable. (Def. Farber Reply at 9).

The AAP job codes are assigned pursuant to a United States Department of Labor regulation:

41 C.F.R. § 60-2.12(b). Since Goldman Sachs was required to classify its employees into AAP job groups precisely on the basis of both similarity of duties and responsibilities and opportunities for compensation and promotion, the defendants cannot now reject that classification system as an input in creating a model for calculating expected pay and advancement. To be sure, the defendants may be able to identify a different variable that perhaps leads to a better fit between a model and the data, but that is an issue that goes to the weight of the evidence; it does not render Dr. Farber's report inadmissible.

3. Aggregation

The defendants contend that Dr. Farber improperly aggregated data across the firm's myriad business units. This position is based both on Goldman Sachs' view of the appropriate relationship between its firm structure and a statistical model and on two technical statistical arguments.

Goldman Sachs maintains that because its decisions about compensation and promotion are made at the business unit level, it was misleading for Dr. Farber to construct a model that fails to take these units into account. (Def. Farber Memo. at 10-11). Indeed, Goldman Sachs' statistical expert, Dr. Michael Ward, has offered evidence that, if the data is broken down by business unit, it becomes apparent that in many units women are compensated at a higher rate than similarly situated men, and in most units there is no statistically significant difference. (Expert Report of Michael P. Ward, Ph.D. in the Matter of Chen-Oster et al. vs. Goldman, Sachs & Co. and The Goldman Sachs Group, Inc. dated July 3, 2014 ("Ward Report") at 55-61). This is a typical argument in many employment discrimination cases. "In equal employment cases involving an employer with a number of locations or subunits, defendants may argue that the data should be examined separately for each unit, while plaintiffs may pool the data into one or several large samples or focus on a few units in which statistical significance was observed." Joseph L. Gastwirth and Efstathia Bura, Some Important Statistical Issues Courts Should Consider in Their Assessment of Statistical Analyses Submitted in Class Certification Motions: Implications for Dukes v. Wal-Mart, 10 Law, Probability and Risk 225, 225 (2011).

Here, Dr. Farber's decision to pool the data across the three Goldman Sachs' divisions at issue is fully supportable. The design of a statistical model must necessarily follow the structure of the entity being studied in light of the questions sought to be answered. In this case, there is substantial evidence that Goldman Sachs utilizes 360 review and quartiling in each division and every business unit. Therefore, it is appropriate to examine these policies across the entire population, even if the effects of those policies may vary in different business units, and even if those effects may not be statistically significant in many individual units.

The difficulty with analyzing data on the business unit level, as Goldman Sachs advocates, is that such disaggregation tends to mask common mechanisms because the sample size in each unit is so small. As a general matter, "[statistical] power increases with sample size. A gender gap of a specific magnitude (say 15%) is more likely to show up as statistically significant in a regression analysis based on 1,000 observations than in one based on 100 observations." William T. Bielby and Pamela Coukos, "Statistical Dueling" with Unconventional Weapons: What Courts Should Know About Experts in Employment Discrimination Class Actions, 56 Emory L.J. 1563, 1597 (2007). Bielby and Coukas demonstrate the significance of this principle using simulated data. They generated theoretical earnings reflecting a gender gap of 11 percent across all offices within a hypothetical firm. Id. at 1599. But when separate regressions are run for each office, this gap in pay apparently disappears.

Id. at 1602 (footnotes omitted).

Dr. Farber's decision to aggregate data, then, was reasonable both in light of the evidence that Goldman Sachs applies common performance measures that influence pay and promotion across business units and in light of the statistical pitfalls of disaggregation.

Nevertheless, the defendants argue that the impropriety of combining data from all three divisions and for lateral hires as well as employees promoted from within the ranks is demonstrated by application of statistical standards known collectively as the "Chow test."7 (Def. Farber Memo. at 18-20). This test, developed by Professor Gregory C. Chow, is designed to determine the extent to which statistical relationships are identical across different populations or within the same population over different time periods. Gregory C. Chow, Tests of Equality Between Sets of Coefficients in Two Linear Regressions, 28 Econometrics 591, 591 (1960).

Id. In common and, perhaps, imprecise parlance, the Chow test determines whether data can be pooled. Goldman Sachs maintains that because combining data across divisions and combining lateral and non-lateral hires "fails" the Chow test, Dr. Farber's model must be rejected.

But the case law does not support the contention that passing the Chow test is an absolute prerequisite for including different data sets in the same regression. See, e.g., Taylor v. District of Columbia. Water & Sewer Authority, 241 F.R.D. 33, 43 (D.D.C.2007) (rejecting claim that aggregating data across different pay plans was improper because it did not satisfy Chow test); Dukes v. Wal-Mart Stores, Inc., 222 F.R.D. 137, 157-58 (N.D.Cal.2004), aff'd, 509 F.3d 1168 (9th Cir.2007), aff'd in part and remanded in part en banc, 603 F.3d 571 (9th Cir.2010), rev'd on other grounds, ___ U.S. ___, 131 S.Ct. 2541, 180 L.Ed.2d 374 (2011); Rossini v. Ogilvy & Mather, Inc., 615 F.Supp. 1520, 1522-23 (S.D.N.Y.1985), vacated and remanded on other grounds, 798 F.2d 590 (2d Cir.1986) (holding "categorical rejection" of regression analysis combining data for two different periods not warranted despite adverse Chow test); Vuyanich v. Republic National Bank of Dallas, 505 F.Supp. 224, 299, 314 (N.D.Tex.1980), vacated on other grounds, 723 F.2d 1195 (5th Cir.1984) (finding aggregation to be "genuine conflict between the experts" notwithstanding failure of plaintiffs model to meet Chow test); but see Reed Construction Data, Inc. v. McGraw-Hill Companies, Inc, 49 F.Supp.3d 385, 405-07 (S.D.N.Y.2014) (excluding expert report that aggregated data but failed Chow test). Where there is a rational basis for aggregation, as there is here, the Chow test is not a basis for categorically rejecting a model that does not meet its requirements.

There are also sound statistical reasons for being cautious about using the Chow test alone to reject a model. In effect, the Chow test measures whether the coefficients of all variables in separate regressions are the same. Efstathia Bura, Joseph L. Gastwirth, and Hiroyuki Hikawa, The Use of Peters-Belson Regression in Legal Cases at 13, available at http://ssrn.com/abstract=1721123 (last visited March 9, 2015). Thus, an analysis could fail the Chow test even if the coefficient for the variable of interest was identical in each regression on disaggregated data, if the coefficients for other variables were dissimilar. Id. For example, regressions might indicate that education or experience has a different impact on compensation, that is, has a different coefficient in each business unit regression, but that gender has a consistent impact. Yet, in this hypothetical, the Chow test would counsel against combining business units in the same regression. More importantly, an analysis could fail the Chow test where the coefficient for the variable of interest varies in magnitude but not in the direction of impact. For instance, if being female is always negatively associated with compensation, even if not always to the same magnitude, it may be appropriate to utilize a unitary model notwithstanding failure to satisfy the Chow test. See id. at 14-15. The fact that a Chow test might indicate that the coefficients for variables conducted on subgroups of the putative class are not equal therefore does not render Dr. Farber's opinion inadmissible.

Finally, Goldman Sachs complains that "Dr. Farber's models have very poor explanatory power." (Def. Farber Reply at 6). Specifically, the defendants note that the correlation coefficients for Dr. Farber's models range from 0.23 to 0.33, meaning that the predictive variables he has utilized explain only about one-quarter to one-third of the differences in compensation in the population of employees studied. (Def. Farber Reply at 6 n. 7). But the correlation coefficient is simply a normalized form of the standard error of estimate, and "a large standard error of estimate does not tell one anything at all about the accuracy with which the effects of the independent variables are measured." Fisher, supra at 71920. In other words, "a large standard error of estimate of the regression tells you that you do not know everything. This is not the same as telling you that you do not know anything." Id. at 720. Thus, what is important to Dr. Farber's analysis is that certain predictive variables have a statistically significant impact on compensation, not that a large portion of the variation in compensation remains unexplained.

For these reasons, the defendants' motion to exclude Dr. Farber's evidence is denied.

B. Dr. Michael P. Ward

The plaintiffs seek to exclude one aspect of the expert report of Dr. Ward, Goldman Sachs' statistical expert. Dr. Ward is Vice President and Senior Analyst at Welch Consulting, a firm specializing in economic and statistical research. (Ward Report at 1). Dr. Ward received his B.A. in economics from the University of California, Santa Barbara, and his M.A. and Ph.D. degrees in economics from the University of Chicago. (Ward Report at 1). He taught economics and statistics at Santa Barbara from 1973 to 1975 and at the University of California, Los Angeles from 1975 to 1980. (Ward Report at 1). From 1980 to 1984 he was employed as a Senior Analyst at the Rand Corporation, after which he joined Welch Consulting. (Ward Report at 1).

The plaintiffs do not seek to exclude the regressions performed by Dr. Ward, which constitute the bulk of his expert report. Rather, they challenge two exercises that he conducted which are characterized as matching pair analyses. (Memorandum of Law in Support of Plaintiffs' Motion to Exclude Expert Testimony of Dr. Michael P. Ward at 1). Goldman Sachs denominates these studies the "Farber-Flaw" analysis and the "Investigative Pairs" analysis. (Defendants' Memorandum in Opposition to Plaintiffs' Motion to Exclude Expert Testimony of Dr. Michael P. Ward ("Def. Ward Memo.") at 6-9). The first consisted of comparing two pairs of male Vice Presidents who shared similar characteristics according to the factors utilized by Dr. Farber in his regressions, but who worked in different business units. (Ward Report at 24-25). Although Dr. Farber's model would have predicted that the two employees in each pair would have similar compensation, in fact their earnings were very different. (Ward Report at 24-25). Dr. Ward then interviewed the. Vice Presidents' managers and determined that there were additional factors not captured by Dr. Farber's model, such as the nature of the business unit and the employee's specific job function, that could explain the discrepancies. (Ward Report at 24-25). According to Goldman Sachs, this is significant because "if the lower-paid Vice President in this pair had been female, Dr. Farber would have erroneously attributed the compensation difference between them to gender." (Def. Ward Memo. at 7 (emphasis omitted)).

Dr. Ward's Investigative Pairs analysis examined pairs of Vice Presidents who, without accounting for gender, shared similar characteristics according to the factors he used in his own regressions — business unit, department, job junction, tenure at Goldman Sachs, quartile, and whether the employee was hired laterally — but who had very different compensation. (Ward Report at 49). Dr. Ward then interviewed the employees' managers to try to ascertain what could account for the pay discrepancies; some of these managers submitted declarations that are appended to Dr. Ward's report. (Ward Report at 49-52 & App. F). From this exercise, Dr. Ward concluded that there are factors that reflect an employee's value to the firm and explain pay differences but are not captured in computerized data and cannot be included in a regression analysis. (Ward Report at 52).

As statistical analyses, Dr. Ward's matched pairs studies are unreliable for a number of reasons. First, he has made no effort to determine the statistical significance of his findings, and it is self-evident that his sample size of two pairs in the "Farber-Flaw" exercise is too small to serve as the basis for any statistical inference. Second, the selection of which matched pairs were analyzed was subject to bias. Although Dr. Ward may have utilized objective criteria in determining which employees "matched," he submitted "dozens" of such pairs to defense counsel, who then chose which pairs were to be examined further. (Deposition of Michael P. Ward, Ph.D., dated Jan. 6, 2014 ("Ward Dep."), excerpts attached as Exh. 3 to Declaration of Cara A. Greene dated July 29, 2014, at 265-67, 310). Finally, there is a significant danger of reporting bias, particularly on the part of the managers who were effectively asked to justify their own decisions to compensate seemingly similar employees differently. A manager may well be expected to attribute such differential treatment to qualities like "leadership potential" rather than gender. See Arredondo v. Delano Farms Co., 301 F.R.D. 493, 512 (E.D.Cal.2014) (finding expert's statistical analysis of alleged wage and hour violations that only evaluated declarations submitted by plaintiffs to be "based upon a limited pool of information" and irrelevant to class certification); In re Front Loading Washing Machine Class Action Litigation, No. 08 Civ. 51, 2013 WL 3466821, at *7 (D.N.J. July 10, 2013) (excluding unreliable consumer survey); York v. Starbucks Corp., No. 08 Civ. 7919, 2011 WL 8199987, at *12-13 (C.D.Cal. Nov. 23, 2011) (excluding survey of class members that "failed to comply with well established principles and procedures designed to insure the reliability of survey data").

Goldman Sachs contends that these arguments are beside the point because the matched pairs analyses were only designed to be "illustrative," and they were not intended to be the basis for any statistical inference. (Def. Ward Memo. at 8; Ward Dep. at 267-68). But a witness must either have first-hand knowledge of the matter about which he testifies, and so testify as a percipient witness, or he must utilize expertise in order to aid the finder of fact in understanding esoteric or complex evidence. See United States v. Mejia, 545 F.3d 179, 194-96 (2d Cir.2008) (finding that "[t]estimony is properly characterized as `expert' only if it concerns matters that the average juror is not capable of understanding on his or her own"); Schwartz v. Fortune Magazine, 193 F.R.D. 144, 147 (S.D.N.Y.2000) (excluding testimony of accounting expert not based on specialized expertise but only on "basic calculations"). Here, Dr. Ward falls into neither category. He disavows using his expertise as a statistician in conducting the matched pairs analysis, and it is the employees and managers who have first-hand knowledge of the compensation-related information that Goldman Sachs proffers. It is inappropriate to present that evidence with the imprimatur of an expert witness. Accordingly, the plaintiffs' motion to exclude Dr. Ward's matched pairs analyses is granted. The defendants, however, are not precluded from relying on the underlying factual matter to the extent it is otherwise admissible.

C. Dr. Wayne F. Cascio

Next, Goldman Sachs seeks to exclude the expert testimony of Dr. Wayne F. Cascio. Dr. Cascio has been the Robert H. Reynolds Distinguished Chair in Global Leadership in the Graduate School of Business Administration at the University of Colorado Denver since 1981. (Expert Report in the Matter of Chen-Oster, et al. v. Goldman Sachs & Co. dated Feb. 18, 2014 ("Cascio Report")8, ¶ 1). He received his Ph.D. in industrial and organizational psychology from the University of Rochester in 1973 and subsequently taught at a number of universities before receiving his current appointment. (Cascio Report, ¶ 3).

Dr. Cascio was engaged by plaintiffs' counsel to evaluate the performance assessment and compensation recommendation processes at Goldman Sachs. (Cascio Report, ¶ 12). He concluded that the measures lack reliability, clear standards for decision making, and proper accountability systems, and were not adequately implemented. (Cascio Report, ¶¶ 16-17). Further, he determined that the processes lacked procedural safeguards. (Cascio Report, ¶ 19). Accordingly, Dr. Cascio found that Goldman Sachs' policies do not meet the basic standards of practice in the field of industrial and organizational psychology and do not justify the resulting gender differences in compensation. (Cascio Report, ¶ 21).

In order to be admissible under Rule 702 and Daubert, an expert opinion must be "relevant to the task at hand," Amorgianos, 303 F.3d at 265, and the task here is to determine the appropriateness of certifying a class. The defendants argue that Dr. Cascio makes no attempt to draw a causal connection between the flaws he identifies in Goldman Sachs' evaluation processes and gender disparities in compensation, an issue that would be relevant to determining whether the commonality requirement for class certification has been met. (Defendants' Memorandum of Law in Support of Motion to Strike Reports, Opinion, and any Testimony of Plaintiffs' Expert Dr. Wayne F. Cascio at 2, 9-10). It is certainly true that Dr. Cascio has disavowed making any such causal connection. (Deposition of Wayne Cascio dated Nov. 16, 2013, excerpts attached as Exh. 4 to Declaration of Barbara B. Brown dated June 13, 2014, at 149, 181-82). But it is not necessary for each expert to provide evidence establishing every element of a party's case, and the plaintiffs have proffered Dr. Farber's analyses in order to prove causation.

Dr. Cascio's opinion is directed instead to rebutting any claim by Goldman Sachs that its evaluation processes must be allowed even if they have a disparate impact on women. Under Title VII,

42 U.S.C. § 2000e-2(k)(1)(A)(i). Thus, expert evidence that addresses the reliability of Goldman Sachs' evaluation processes goes to the business necessity defense and is relevant at the class certification stage to the question of commonality. See McReynolds v. Merrill Lynch, Pierce, Fenner & Smith, Inc., 672 F.3d 482, 489 (7th Cir.2012) (finding that whether "teaming" policy is justified by business necessity is an issue "common to the entire class and therefore appropriate for class-wide determination"); Houser v. Pritzker, 28 F.Supp.3d 222, 243 (S.D.N.Y.2014) (holding commonality established by defenses applicable to all class members). It is true, as the defendants suggest, that the merits of Dr. Cascio's opinion will only be tested at trial if the plaintiffs first carry their burden of demonstrating that the firm's evaluative processes have a disparate impact. (Defendants' Reply Memorandum in Support of Their Motion to Strike Reports, Opinion, and Any Testimony of Plaintiffs' Expert Dr. Wayne F. Cascio at 7-8). But that does not make his evidence any less relevant at the class certification stage, as the cases cited above demonstrate. Accordingly, Goldman Sachs' motion is denied.

D. Dr. Michael A. Campion

Dr. Michael A. Campion has a Ph.D. in industrial and organizational psychology and has been a Professor of Management at Purdue University since 1986. (Report of Michael A. Campion, Ph.D. on Performance Evaluation (360 Feedback and Quartiling) and Incentive Bonus Compensation in Chen-Oster v. Goldman Sachs dated Dec. 11, 2013 ("Campion Report"), ¶ 4). Prior to joining the Purdue faculty, Dr. Campion worked for four years each at IBM Corporation and Weyerhaeuser Company as a human relations and management analyst. (Campion Report, ¶ 7).

Dr. Campion's expert report on behalf of Goldman Sachs is effectively a mirror image of the report submitted by Dr. Cascio. Dr. Campion concluded that the 360 review and quartiling processes utilized by Goldman Sachs are valid and conform with best practices. (Campion Report, ¶¶ 14-20). He also found that not only is providing incentive bonuses valid, it is a best practice and is particularly common in the fmancial services industry. (Campion Report, ¶¶ 21-23). Finally, Dr. Campion took issue with Dr. Cascio's criticisms of Goldman Sachs' performance evaluation techniques. (Campion Report, ¶ 24).

The plaintiffs assert that Dr. Campion's analysis is flawed in two respects. First, they contend that his methodology for determining best practices — essentially a meta-analysis of journal articles concerning performance evaluation techniques — is inherently unsound and that, in any event, he applied it to Goldman Sachs' policies in an unreliable manner. (Memorandum of Law in Support of Plaintiffs' Motion to Exclude Expert Testimony of Dr. Michael A. Campion ("Pl. Campion Memo.") at 1, 10-13; Campion Report, ¶¶ 15, 19, 32-35, 51-54, 59-63). Second, the plaintiffs question the validity of Dr. Campion's suggestion that Dr. Cascio inferred bias merely from differential performance evaluations, without taking into account potential gender differences such as relative competitiveness in "extreme jobs" such as those held by professionals at Goldman Sachs. (Pl. Campion Memo. at 1, 6-8; Campion Report, ¶¶ 100-104).

Whatever the merit of the plaintiffs' arguments, they go exclusively to Dr. Campion's opinions on the merits of their claims and not to issues of class certification. Accordingly, it is premature to assess the plaintiffs' challenges, and their motion is denied.

E. Michael P. Curran

Michael P. Curran is a Director of Towers Watson, a business consulting firm, where he specializes in providing advice concerning compensation and related matters to financial services firms. (Report of Michael P. Curran in Chen-Oster, et al. v. Goldman, Sachs & Co., et al. ("Curran Report"), ¶ 1). After receiving a B.S. degree in industrial & labor relations from Cornell University in 1979, Mr. Curran worked in the paper industry and for Morgan Stanley until 1983, when he joined McLagan Partners, a consulting firm focused on the financial services industry. (Curran Report, ¶¶ 2-4). In 2003, he became President of MGMC, a boutique compensation consulting firm serving banks and investment banks. (Curran Report, ¶ 5). When the corporate predecessor to Towers Watson acquired MGMC in 2007, Mr. Curran became the head of global finance for the firm. (Curran Report, ¶ 6).

Mr. Curran submitted an expert report on behalf of Goldman Sachs in which he expressed opinions about the structure of compensation in the financial services industry. Specifically, he concluded that (1) Associates and Vice Presidents in the industry hold a wide variety of jobs, which differ by type of product, activity of the employer, required skill set, and nature of the clients served; (2) compensation varies widely, depending on the success of the firm and of the employee's business unit, individual productivity, and market demand; (3) lateral hiring can have a significant impact on compensation; (4) managers in the industry typically and necessarily exercise significant judgment in setting compensation; and (5) firms in the industry strive to link compensation with each employee's contribution. (Curran Report, ¶¶ 11-15).

The plaintiffs seek to exclude Mr. Curran's evidence on two grounds. First, they contend that it is irrelevant because it relates to the financial services industry as a whole rather than to Goldman Sachs specifically. (Memorandum of Law in Support of Plaintiffs' Motion to Exclude Expert Testimony of Michael P. Curran ("Pl. Curran Memo.") at 9-10; Reply Memorandum of Law in Support of Plaintiffs' Motion to Exclude Expert Testimony of Michael P. Curran ("Pl. Curran Reply") at 2-4). Second, the plaintiffs argue that Mr. Curran's opinions are unreliable in that they are based on Towers Watson surveys, the validity of which cannot be tested because the defendants have not disclosed the data underlying those studies. (Pl. Curran Memo. at 5-7; Pl. Curran Reply at 4-7).

The first of these arguments is fully dispositive for present purposes. The defendants assert that "Mr. Curran's testimony is relevant not to show what Goldman Sachs does, but rather to show practices in the financial services industry generally...." (Defendants' Memorandum in Opposition to Plaintiffs' Motion to Exclude Expert Testimony of Michael P. Curran ("Def. Curran Memo.") at 7). It is certainly the case, as Goldman Sachs contends, that expert testimony is admissible to show industry practice. (Def. Curran Memo. at 7). See Reach Music Publishing, Inc. v. Warner Chappell Music, Inc., 988 F.Supp.2d 395, 403 (S.D.N.Y.2013); Teevee Toons, Inc. v. Rep Sales, Inc., No. 03 Civ. 10148, 2007 WL 5011652, at *1 (S.D.N.Y. Oct. 19, 2007). But industry practice must itself be relevant to the issues under consideration. See Sompo Japan Insurance Co. of America v. Norfolk Southern Railway Co., 762 F.3d 165, 180-81 (2d Cir.2014) (upholding exclusion of evidence of industry practice found irrelevant); Travelers Indemnity Co. v. Northrop Grumman Corp., No. 12 Civ. 3040, 2014 WL 464769, at *5 (S.D.N.Y. Jan. 28, 2014) (excluding expert testimony when industry practice "not an issue in this case"). Here, industry practice concerning compensation might ultimately be relevant, for example, to support a business necessity defense. However, it is not pertinent to class certification. What is important at this stage is whether there are common issues relating to Goldman Sachs and the claims against it. As Mr. Curran does not purport to address such issues, his evidence is excluded.

F. Dr. David L. Yermack

Dr. David L. Yermack is the Albert Fingerhut Professor of Finance and Business at New York University's Stern School of Business. (Rebuttal Report of David L. Yermack, Expert Witness for Plaintiffs ("Yermack Report"), ¶ 2). He received a Bachelor of Economics, M.B.A., J.D., Master of Business Economics, and Ph.D. in business economics, all from Harvard University. (Yermack Report, ¶ 2). After receiving his doctorate, he immediately joined the faculty of the Stern School, and he received tenure in 2001. (Yermack Report, ¶ 2).

Dr. Yermack's expert report is a rebuttal to the opinions of Mr. Curran. In summary, he concludes that financial services firms have similar ranges of jobs and that staff at the Associate and Vice President level are comparable across firms. (Yermack Report, ¶ 13). According to Dr. Yermack, the compensation of such employees is far less variable than that of more senior professionals. (Yermack Report, ¶ 14). Similarly, the link between compensation and the employee's perceived contribution to the firm is stronger at the more senior levels. (Yermack Report, ¶ 17).

Goldman Sachs challenges Dr. Yermack's evidence in several respects. It argues that to the extent Dr. Yermack is offering an opinion about the diversity of jobs at Goldman Sachs, this is not a proper subject for expert testimony. (Defendants' Memorandum in Support of Their Motion to Exclude Expert Testimony of David L. Yermack ("Def. Yermack Memo.") at 5-6; Defendants' Reply Memorandum in Support of Their Motion to Exclude Expert Testimony of David L. Yermack ("Def. Yermack Reply") at 2-3). The defendants further contend that, as an academician, Dr. Yermack lacks the expertise to opine on the "nuts and bolts" of the financial industry. (Def. Yermack Memo. at 6-7; Def. Yermack Reply at 4-6). Finally, Goldman Sachs posits that Dr. Yermack's conclusions are not based on sufficient facts and data. (Def. Yermack Memo. at 7-13; Def. Yermack Reply at 7-9).

For present purposes, these issues are moot. The plaintiffs acknowledge that Dr. Yermack is a rebuttal expert, whose evidence will only be offered to the extent it is necessary to controvert the opinions of Mr. Curran. (Pl. Curran Reply at 1 n. 1). Since those opinions have been excluded from consideration in connection with the motion for class certification, Dr. Yermack's shall be as well. If appropriate, these questions will be revisited at the merits stage.

Non-Expert Evidence

Finally, the plaintiffs move to strike declarations submitted by Goldman Sachs managers as well as an exhibit identified as Appendix A and attached to the defendants' memorandum in opposition to class certification. I will address these issues separately.

A. Manager Declarations

In opposing the class certification motion, Goldman Sachs submitted a number of declarations by firm managers to support the arguments that Goldman Sachs professionals hold a variety of disparate positions in many different business units and that managers exercise broad discretion in making compensation and promotion decisions within each unit. The plaintiffs seek to exclude most of those declarations9 on the ground that Goldman Sachs breached its disclosure obligations by failing to identify the declarants prior to proffering their statements. This application is without merit.

Pursuant to Rule 37(c)(1) of the Federal Rules of Civil Procedure,

Rule 26(a), in turn, requires that, at the outset of the litigation, each party must identify "each individual likely to have discoverable information," while Rule 26(e) creates an obligation to supplement those disclosures as additional information becomes available. In assessing whether preclusion is warranted under Rule 37(c), a court must consider

Patterson v. Balsamico, 440 F.3d 104, 117 (2d Cir.2006) (internal quotation marks and citation omitted) (alterations in original).

Here, it is doubtful that Goldman Sachs breached any disclosure obligation in the first place. Rule 26(a)(1)(A)(i) requires identification of persons that a party "may use" to support its claims or defenses, not everyone with knowledge about the subject matter. See Fed.R.Civ.P. 26(a)(1) advisory committee's note (2000); Government Benefits Analysts, Inc. v. Gradient Insurance Brokerage, Inc., Civ. A. No. 10-2558, 2012 WL 3292850, at *5-6 (D.Kan. Aug. 13, 2012). In a case such as this that concerns firm-wide structure and policies, every manager, and, indeed, every employee, is a potential witness; yet it would hardly further the purpose of the rule if counsel were to produce a roster of the entire firm as part of its initial disclosure. To the extent that the plaintiffs argue that Goldman Sachs should have supplemented its initial disclosure at some subsequent point when it had decided whose declarations it was going to use, the argument is foreclosed by the plaintiffs' own conduct. They rejected the defendants' suggestion that the parties exchange draft declarations or lists of declarants at some time in advance of the time that the declarations were filed. (Letter of Barbara Berish Brown and Theodore O. Rogers, Jr., dated Oct. 18, 2013, attached as Exh. 3 to Declaration of Theodore O. Rogers, Jr., dated Aug. 12, 2014 ("Rogers 8/12/14 Decl.")). Furthermore, the plaintiffs themselves never identified their witnesses prior to filing their declarations. The plaintiffs are bound by this (at least implicit) agreement not to identify the declarants in advance.

Even if the defendants had violated the rules, preclusion would not be warranted in light of the factors enumerated in Patterson. First, for the reasons already discussed, Goldman Sachs has substantial justification for any "non-compliance." Next, the evidence of Goldman Sachs' declarants is plainly important: the firm's structure and its compensation and promotion policies are central to a determination of issues such as commonality and predominance. Third, the plaintiffs suffered no prejudice, since they could have taken the depositions of the Goldman Sachs declarants but eschewed that opportunity when they did not receive all of the documentary information they had demanded. (Rogers 8/12/14 Dect., Exhs. 5-15). Finally, an extension of the class certification briefing schedule to accommodate additional discovery with respect to the declarants, while hardly ideal, would have been feasible. Accordingly, the motion to exclude the declarations of the Goldman Sachs managers is denied.

B. Appendix A

The document identified as Appendix A is a schematic of the three Goldman Sachs divisions at issue in the class certification motion and subsidiary business units. Again, the plaintiffs seek to preclude it on the ground that it was not previously disclosed. (Pl. Preclusion Memo. at 10-11). They also argue that it in fact misrepresents the underlying facts that it purports to illustrate because it conflates the firm's corporate structure at different points in time into a single visual representation. (Reply Memorandum of Law in Support of Plaintiffs' Motion to Strike the Declarations of Susan Benz, Jean Marie Bohm, Donald Casturo, Darren Cohen, Stephanie Cohen, Nora Creedon, Samantha Davidson, Celeste Guth, Marie Louise Kirk, John Levene, Todd Lopez, James McNamara, Craig Packer, Stephen Pierce, Laura Robertson, Peter Craig Russell, Susan Jane Scher, and Megan Taylor, and to Exclude Appendix A to Defendants' Memorandum of Law in Opposition to Plaintiffs' Motion for Class Certification at 7-8).

Appendix A is essentially a demonstrative exhibit; it has no probative value apart from the underlying evidence on which it is based. As such, it does not fit comfortably within the disclosure requirements of Rule 26(a)(1)(A)(ii), since it would not normally exist at the outset of the case. And, to the extent that the plaintiffs argue that the exhibit is misleading, this is a contention that may be considered in connection with its weight rather than its admissibility, since class certification issues are heard by the court rather than by a jury. The application to preclude Appendix A is therefore denied.

Conclusion

For the reasons discussed above, the motion to exclude the expert testimony of Dr. Farber (Docket no. 356) is denied; the motion to exclude in part the expert testimony of Dr. Ward (Docket no. 307) is granted; the motion to exclude the expert testimony of Dr. Cascio (Docket no. 350) is denied; the motion to exclude the expert testimony of Dr. Campion (Docket no. 303) is denied; the motion to exclude the expert testimony of Mr. Curran (Docket no. 305) is granted; the motion to exclude the expert testimony of Dr. Yermack (Docket no. 323) is granted; and the motion to preclude the declarations of Goldman Sachs' managers and Appendix A to Goldman Sachs' memorandum in opposition to class certification (Docket no. 299) is denied. It bears repeating that these determinations are made solely with respect to the submission of this evidence in connection with the class certification motion; admissibility in connection with dispositive motions or trial is subject to further analysis.

SO ORDERED.