This case presents a question of first impression in this Circuit--whether Fed.R.Evid. 702 permits a defendant in a criminal prosecution to adduce, from an expert in the field of human perception and memory, testimony concerning the reliability of eyewitness identifications. The district court refused to admit the testimony of a psychologist offered by the defendant, apparently because the court believed that such testimony can never meet the "helpfulness" standard of Fed.R.Evid. 702. We hold that the district court erred. We also hold that the admission of such expert testimony is not automatic but conditional. First, the evidence must survive preliminary scrutiny in the course of an in limine proceeding conducted by the district judge. This threshold inquiry, which we derive from the helpfulness standard of Rule 702, is essentially a balancing test, centering on two factors: (1) the reliability of the scientific principles upon which the expert testimony rests, hence the potential of the testimony to aid the jury in reaching an accurate resolution of a disputed issue; and (2) the likelihood that introduction of the testimony may in some way overwhelm or mislead the jury. Second, admission depends upon the "fit," i.e., upon a specific proffer showing that scientific research has established that particular features of the eyewitness identifications involved may have impaired the accuracy of those identifications. The district court's assessment of these factors will guide its discretion in deciding whether to admit the evidence under Fed.R.Evid. 702, which contemplates a liberal view toward the admissibility of expert testimony generally. The district court's ruling under Fed.R.Evid. 702 will be reviewable under an abuse of discretion standard. Finally, the district court retains discretionary authority under Fed.R.Evid. 403 to exclude any relevant evidence that would unduly waste time or confuse the issues at trial.

The defendant in this case was convicted solely on the basis of eyewitness testimony. On the record before us the error cannot be deemed harmless, and accordingly we will vacate the judgment of conviction. It is possible that the district court might conclude, after holding a foundation hearing required by Fed.R.Evid. 702 in this case, that the proffered expert testimony is inadmissible in which case a new trial would be unnecessary. We therefore remand for such a hearing on admissibility, cf. Waller v. Georgia, --- U.S. ----, 104 S. Ct. 2210, 2217, 81 L. Ed. 2d 31 (1984); United States v. Faison, 679 F.2d 292, 297-98 (3d Cir.1982), with the proviso that, if the evidence is found admissible, the district court will have to order a new trial.

Appellant, John W. Downing, was indicted for mail fraud, 18 U.S.C. Secs. 1341 & 1342; wire fraud, 18 U.S.C. Sec. 1343; and interstate transportation of stolen property, 18 U.S.C. Sec. 2314. All counts of the indictment arose from a scheme to defraud numerous vendors conducted in 1978 and 1979 by a group of individuals calling themselves the Universal League of Clergy (U.L.C.). U.L.C. operated first out of Bedford, Massachusetts, and then out of Blue Bell, Pennsylvania. Its modus operandi was essentially as follows. U.L.C. representatives at national trade shows made contact with manufacturers' representatives by expressing an interest in their product line. When the representative took an order for his product, U.L.C. furnished the vendor with the list of supposed credit references. In fact, U.L.C. had compiled a list of non-existent trade and bank references; the trade addresses were actually the addresses of mail-drops, and the bank reference was a foreign post office box. Later, when the credit department of the manufacturer investigated the "references"--usually by mail--it received favorable reports concerning U.L.C.'s payment history with other corporate creditors and assurances from the "bank" that U.L.C.'s account was substantial. These reports, the government established, were actually supplied by U.L.C. itself after collecting the credit inquiries from the various mail-drops. These positive credit references usually induced the manufacturer to ship goods to U.L.C. on credit. U.L.C. then disposed of the goods, without making payment to the manufacturers.

U.L.C. was represented in these dealings by men identifying themselves as U.L.C. clergy, including "Reverend" or " 'Doctor' Claymore," "Malcolm Sloane," "Reverend Olson," "Paul Eaton," and "Richard Thomas." The government contended that the individuals acting as the U.L.C. clergy were actually appellant and his co-defendants, James A. Silva and Richard Piazza. The central issue at the trial was the identification of appellant as Reverend Claymore. Silva and Piazza admitted setting up U.L.C., but denied knowing that the suppliers were going to be defrauded. They asserted that they were innocent dupes of Reverend Claymore, who masterminded the entire scheme. They (along with appellant) further asserted that appellant was not Claymore, and that if the government could only find the real Claymore, their innocence would be proved.1

The government's case against appellant consisted primarily of the testimony of twelve eyewitnesses who, with varying degrees of confidence, testified that appellant was the man they knew as Reverend Claymore.2 These witnesses testified on the basis of their personal observations of Reverend Claymore for periods ranging from 5 to 45 minutes during the course of business dealings that later were discovered to be fraudulent. Appellant contended at trial that these eyewitnesses were mistaken and that their testimony was unreliable because of the short period of time in which the witnesses had to view Claymore, the innocuous circumstances of their meetings with him, and the substantial lapse of time between the meetings and the subsequent identifications.

In an effort to overcome the substantial weight of twelve eyewitness identifications in the jury's mind, appellant's counsel, at the beginning of the trial, inquired whether the court would permit expert testimony on the unreliability of eyewitness testimony. Transcript Vol. 43, at 4-2 to 4-5. The district court deferred ruling on the motion and requested that appellant's counsel inform it during a break in the proceedings as to the substance of the proposed expert testimony, and as to any federal cases that have held such testimony to be admissible. On the tenth day of trial, following an off-the-record side-bar discussion, the court briefly summarized the parties' positions and then denied appellant's motion:

THE COURT: There is a discussion at side bar.

Mr. Concannon offered Robert Weisburg, Ph.D. of Temple University, who is a cognative [sic] psychologist. He is an assistant professor of psychology at Temple University.

Mr. Concannon wants to offer him as an expert witness in order to present him to the Jury as a person who would be able to help them deal with the problem of identification of the defendants or any witnesses.

He also would want him to answer a hypothetical question which would be related to the evidence presented in this case.

The U.S. Attorney's position is this would usurp the function of the Jury, and he opposes the witness appearing.

Mr. Concannon said the objection the U.S. Attorney has could be dealt with as of cross examination.

Anything else you want to add to that summary?

MR. CONCANNON: This is basically it, Your Honor.

THE COURT: Anything you want to add to what has been said?

MR. SCHENCK: No, but I have case law.

THE COURT: Do you want to cite any cases?

MR. SCHENCK: The United States versus Focher [sic], 590 Fed.2d 381, 1979 case by the Third Circuit [sic].

In a recent case with Judge Green of this court he declined to admit the same type of evidence.

THE COURT: Anything else?

MR. SCHENCK: No.

THE COURT: It is the ruling of this court that the motion to have the psychologist testify is denied because this is a function of the jury to deal with the credibility of the witness[es] that have appeared here and give whatever weight to that testimony that they see fit and also determine if their evidence is credible. What we have in this case is not only a matter of identification, but we have as an [sic] additional evidence such as finger prints, handwriting, which is also for the jury and the credibility of the experts that appeared.

Transcript, Vol. 55, at 3-4.

This dialogue represents the total on-the-record discussion concerning the admissibility of defendant's expert testimony. The case went to the jury without the expert's testimony and appellant was convicted. He appeals, asserting that the district court's exclusion of his expert's testimony was erroneous and harmful within the meaning of Fed.R.Evid. 103(a).

As the transcript of the colloquy indicates, the district court articulated two reasons for refusing to permit appellant's expert witness to testify: (1) the witness would usurp the "function of the jury"; and (2) there was additional evidence "such as fingerprints [and] handwriting." We note at the outset, and the government concedes, that the court was in error as to the second ground: no fingerprint or handwriting evidence was offered against appellant; rather, the government's case rested almost exclusively on the eyewitness identifications.

The first ground for the court's decision does not proceed from a similar misapprehension of the record, but the court's reasoning in this regard does lack clarity. Initially, it would appear that the court was concerned that the expert witness would testify as to the "ultimate issue of fact," Fed.R.Evid. 704. Were this so, the first ground of decision would also be erroneous. As the advisory committee's note on Rule 704 points out, the basic approach to opinion testimony in the federal rules is one of helpfulness. "In order to render this approach fully effective and to allay any doubt on the subject, the so-called 'ultimate issue' rule is specifically abolished by [Rule 704]." Notes of Advisory Committee on Proposed Rule 704.3 The rule rejects as "empty rhetoric" the notion that some testimony is inadmissible because it usurps the "province of the jury." Id. (citing 7 J. Wigmore, Treatise on the Anglo-American System of Evidence in Trials at Common Law Sec. 1920, at 17 (3d ed. 1940)).

In light of this clear mandate of Fed.R.Evid. 704, it appears rather that the district court based its ruling on an interpretation of Fed.R.Evid. 702, in effect concluding that expert testimony concerning the reliability of eyewitness identifications is never admissible in federal court because such testimony concerns a matter of common experience that the jury is itself presumed to possess. Under this approach, an expert's testimony on the reliability of eyewitnesses can never meet the test for the admissibility of expert testimony contained in Fed.R.Evid. 702.

If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education, may testify thereto in the form of an opinion or otherwise.

This rule invests trial courts with broad discretion to admit expert testimony over the objection that it would improperly invade the province of the jury. Under Rule 702, "an expert can be employed if his testimony will be helpful to the trier of fact in understanding evidence that is simply difficult, [though] not beyond ordinary understanding." S. Saltzburg & K. Redden, Federal Rules of Evidence Manual 451 (3d ed. 1982). See also Notes of Advisory Committee on Proposed Rule 702 (quoting Ladd, Expert Testimony, 5 Vand.L.Rev. 414, 418 (1952)); 3 J. Weinstein & M. Berger, Weinstein's Evidence Sec. 702, at 702-12 n. 6 (citing cases). Cf. Breidor v. Sears, Roebuck and Co., 722 F.2d 1134, 1138 (3d Cir.1983) (citing Salem v. United States Lines Co., 370 U.S. 31, 35, 82 S. Ct. 1119, 1122, 8 L. Ed. 2d 313 (1962)) (district court has broad discretion to admit or exclude expert evidence, and its action will be sustained unless manifestly erroneous); Knight v. Otis Elevator Company, 596 F.2d 84, 87 (3d Cir.1979) (noting the liberal policy of admitting expert testimony which will "probably aid" the trier of fact).

Notwithstanding the fact that the Rule 702 standard usually favors admissibility, see In re Japanese Electronic Products Antitrust Litigation, 723 F.2d 238, 279 (3d Cir.1983), several courts of appeals have upheld the exclusion of expert testimony on eyewitness perception and memory because the testimony would involve questions that "can be adequately addressed in cross examination and that the jury can adequately weigh ... through common-sense evaluation." United States v. Thevis, 665 F.2d 616, 641 (5th Cir.1982). See also United States v. Fosher, 590 F.2d 381, 383 (1st Cir.1979) (offer of testimony focusing on general problems of eyewitness identification was not "sufficiently beyond the ken of lay jurors to satisfy Rule 702"); United States v. Brown, 540 F.2d 1048, 1054 (10th Cir.1976) (testimony properly refused because it "usurps the functions of the jury," and because the offer of proof was inadequate); United States v. Amaral, 488 F.2d 1148, 1153 (9th Cir.1973) (cross-examination will be adequate to reveal any frailties in eyewitness identifications).4

We have serious doubts about whether the conclusion reached by these courts is consistent with the liberal standard of admissibility mandated by Rule 702.5 Instead, we find persuasive more recent cases in which courts have found that, under certain circumstances, this type of expert testimony can satisfy the helpfulness test of Rule 702. For example, in State v. Chapple, 135 Ariz. 281, 660 P.2d 1208 (1983) (applying Arizona's version of the Federal Rules of Evidence), the Supreme Court of Arizona set aside a jury's guilty verdict and ordered a new trial on the ground that the trial court had erroneously excluded an expert on eyewitness identification offered by the defendant. In addressing the question whether the expert's testimony would have been "helpful" to the jury in reaching an informed decision, the court noted several specific factual "variables" that were present in that case which, the defendant's expert was prepared to testify, reduced the eyewitnesses' ability to perceive and remember accurately.

The proffer stated that the expert would testify concerning: (1) the "forgetting curve," i.e., the fact that memory does not diminish at a uniform rate; (2) the fact that, contrary to common understanding, stress causes inaccuracy of perception and distorts one's subsequent recall; (3) the "assimilation factor," which indicates that witnesses frequently incorporate into their identifications inaccurate information gathered after the event and confused with the event; (4) the "feedback factor," which indicates that where identification witnesses discuss the case with each other they can unconsciously reinforce their individual identifications; and (5) the fact that studies demonstrate the absence of a relationship between the confidence a witness has in his or her identification and the actual accuracy of that identification.6 Each of these "variables" goes beyond what an average juror might know as a matter of common knowledge, and indeed some of them directly contradict "common sense." For this reason, the Arizona Supreme Court concluded that the expert's testimony would have assisted the jury in reaching a correct decision. Id. at 1219-24. See also infra note 24.

In another case, United States v. Smith, 736 F.2d 1103 (6th Cir.1984) (per curiam), the Sixth Circuit held that expert testimony on the reliability of eyewitness identification met the "helpfulness" test of Rule 702 and therefore had been improperly excluded.7 The excluded testimony would have focused on "a hypothetical factual situation identical" to the facts of the case, and would have explained (1) that a witness who does not identify the defendant in a first line-up may "unconsciously transfer" his visualization of the defendant to a second line-up and thereby incorrectly identify the defendant the second time; (2) that studies demonstrate the inherent unreliability of cross-racial identifications; and (3) that an encounter during a stressful situation decreases the eyewitness' ability to perceive and remember and decreases the probability of an accurate identification. Id. at 1105-06. In concluding that this evidence would have assisted the jury in reaching an accurate decision, the court emphasized that the expert's proffered testimony was based upon "the exact facts before the court and not only might have assisted the jury, but might have refuted their otherwise common assumptions about the reliability of eyewitness identification." Id. at 1106 (emphasis in original). See also infra note 24.

The California Supreme Court has recently adopted this view. In People v. McDonald, 37 Cal. 3d 351, 208 Cal. Rptr. 236, 690 P.2d 709 (1984) (summarized in 36 Crim.L.Rep. (BNA) 2201 (Dec. 19, 1984)), the Court held that, under certain narrow circumstances, it will be error for trial courts to exclude qualified expert testimony on eyewitness perception and memory.8 After surveying the "impressive" professional literature in the area, the McDonald court rejected the prosecution argument that section 801(a) of the California Evidence Code, which "limits expert testimony to subjects beyond the range of common experience," cf. Fed.R.Evid. 702, suffices to bar this type of testimony. While conceding the proposition that all jurors know that certain factors, such as lighting, distance, and duration, may affect the accuracy of identifications, the court went on to point out that "[i]t appears from the professional literature, however, that [certain] factors bearing on eyewitness identifications may be known only to some jurors, or may be imperfectly understood by many, or may be contrary to the intuitive beliefs of most." McDonald, supra, 208 Cal.Rptr. at 266, 690 P.2d at 720.

We agree with the courts in Chapple, Smith, and McDonald that under certain circumstances expert testimony on the reliability of eyewitness identifications can assist the jury in reaching a correct decision and therefore may meet the helpfulness requirement of Rule 702. For example, most people, and hence most jury members, probably believe that stress increases the accuracy of one's perception. The expert in the Chapple case, however, proposed to testify that studies reveal precisely the opposite effect. The sole evidence against the defendant in Chapple consisted of the eyewitness identification testimony of two people who viewed the murders under the belief that they might soon become victims as well. In such a setting, the effect of stress on the eyewitnesses was certainly relevant to an evaluation of the reliability of their identifications, and the expert's testimony, if itself reliable, see infra section IV.A., clearly would have assisted the jury.

Judicial resistance to the introduction of this kind of expert testimony is understandable given its innovativeness and the fear of trial delay spawned by the spectre of the creation of a cottage industry of forensic psychologists.9 The logic of Fed.R.Evid. 702 is inexorable, however, and requires, as the Smith, Chapple and McDonald courts recognized, that expert testimony on eyewitness perception and memory be admitted at least in some circumstances. We therefore conclude that the district court erred as a matter of law when it in effect decided that expert testimony on the subject is simply not admissible.

The conclusion that expert testimony on the perception of eyewitnesses, at least in certain cases, meets the helpfulness standard of Rule 702 does not end our analysis. On remand, the district court will still have to decide whether to admit the specific testimony proffered in this case. This exercise of the district court's discretion will be shaped by the policies of Fed.R.Evid. 702, as it impinges on the court's assessment of the scientific basis for the proferred testimony, and by the policies of Fed.R.Evid. 403. The balance of this section is devoted to a discussion of the perceived evidentiary problems posed by novel forms of scientific expertise, generally, and to an analysis of the test announced in Frye v. United States, 293 Fed. 1013 (D.C.Cir.1923), as a way of dealing with those problems. We conclude that the status of the Frye test under Rule 702 is somewhat uncertain, but reject that test for reasons of policy. In section IV, we set forth an alternative standard for evaluating novel scientific evidence that we believe comports with the language and policy of Rule 702. Section V addresses the impact of Fed.R.Evid. 403 on the district court's task on remand.

Courts and commentators have divided over the issue of the foundational requirements for the admission of scientific testimony. Evidence that derives from principles and techniques of uncontroverted validity is, of course, readily admissible, subject to the qualification of the proposed witness and, in some jurisdictions, to a showing that proper safeguards were employed in obtaining the evidence on the relevant occasion.10 Where proffered evidence arises from a novel form of scientific expertise, however, courts and commentators have taken any one of several different approaches to the question of admissibility. First, many courts impose a foundational requirement that the underlying scientific principle or technique be generally accepted in the field to which it belongs. See, e.g., Frye v. United States, 293 Fed. 1013 (D.C.Cir.1923).11

Other courts and commentators have suggested variations on the Frye standard, see, e.g., S. Saltzburg & K. Redden, Federal Rules of Evidence Manual 452 (3d ed. 1982) ("reasonable scientific acceptance"), or alternative approaches. See, e.g., State v. Temple, 302 N.C. 1, 273 S.E.2d 273 (1981) (new scientific evidence can be admitted when its accuracy and reliability have become established and recognized); Giannelli, "The Admissibility of Novel Scientific Evidence: Frye v. United States, A Half Century Later," 80 Colum.L.Rev. 1197, 1249-50 (1980) (criminal defendant and civil litigants should be required to establish the validity of the scientific principle or technique by a preponderance of the evidence; the prosecution in a criminal trial should be required to prove validity beyond a reasonable doubt).

Finally, a third group of courts and commentators reasons that the novelty of the scientific basis for expert testimony bears principally on the weight of the evidence, and that the Federal Rules of Evidence suggest, if not mandate, a generalized "relevancy" approach akin to the balancing test codified in Fed.R.Evid. 403. See United States v. Williams, 583 F.2d 1194, 1198 (2d Cir.1978) (court relies on the "established considerations applicable to the admissibility of evidence"), cert. denied, 439 U.S. 1117, 99 S. Ct. 1025, 59 L. Ed. 2d 77 (1979); State v. Hall, 297 N.W.2d 80 (Iowa 1980) (en banc ); 3 J. Weinstein & M. Berger, Weinstein's Evidence, p 702, at 702-15 to -21.

As is apparent, a considerable amount of judicial and scholarly attention has been devoted to identifying when evidence resting on novel scientific principles or techniques is admissible. Because this court has never adopted a a clear position on this issue, we must decide, based on the language and policies of the Federal Rules of Evidence, the appropriate test for the admissibility of novel scientific evidence.

Our starting point for this discussion is the seminal opinion of Frye v. United States, 293 Fed. 1013 (D.C.Cir.1923), which had won adherents in many state and federal courts at the time of the adoption of the Federal Rules of Evidence. See Giannelli, supra, at 1228-29. The Frye court concluded that expert testimony relating to novel scientific evidence must satisfy a special foundational requirement not applicable to other types of expert testimony. The court stated:

While courts will go a long way in admitting expert testimony deduced from a well-recognized scientific principle or discovery, the thing from which the deduction is made must be sufficiently established to have gained general acceptance in the particular field to which it belongs.

Frye, 293 F. at 1014 (emphasis added). Under Frye, therefore, courts confronted with a proffer of novel scientific evidence must make a preliminary determination through the introduction of other evidence, including expert testimony, regarding (1) the status, in the appropriate scientific community, of the scientific principle underlying the proffered novel evidence; (2) the technique applying the scientific principle; and (3) the application of the technique on the particular occasion or occasions relevant to the proffered testimony. See Giannelli, supra, at 1201. See, e.g., United States v. Brown, 557 F.2d 541, 557 (6th Cir.1977) (applying Frye test to ion microprobic analysis); United States v. Tranowski, 659 F.2d 750, 754-57 (7th Cir.1981) (applying Frye test to testimony by astronomer purporting to date a photograph by measuring lengths of shadows on the photo). Once a novel form of expertise is judicially recognized, this foundational requirement can be eliminated, as is done when, for example, fingerprint, ballistics, or x-ray evidence is offered. See generally Giannelli, supra, at 1248; cf. 1 D. Louisell & C. Mueller, Federal Evidence Sec. 105, 826-27 (1977) (general acceptance standard should be absorbed in judicial notice concept which allows courts to admit scientific evidence without testimony describing the principles resolved); McCormick on Evidence, supra, at 608.

Because the general acceptance standard set out in Frye was the dominant view within the federal courts at the time the Federal Rules of Evidence were considered and adopted, one might expect that the rules themselves would make some pronouncement about the continuing vitality of the standard. Neither the text of the Federal Rules of Evidence nor the accompanying notes of the advisory committee, however, explicitly set forth the appropriate standard by which the admissibility of novel scientific evidence is to be established. Although the commentators agree that this legislative silence is significant, they disagree about its meaning. Professors Saltzburg and Redden, for example, have stated, "[i]t would be odd if the Advisory Committee and the Congress intended to overrule the vast majority of cases excluding such evidence as lie detectors without explicitly stating so." Saltzburg & Redden, supra, at 452. See also 1 D. Louisell & C. Mueller, supra, Sec. 105, at 818 ("Probably the general scientific acceptance approach has survived the enactment of the Federal Rules, and will continue to be applied in determining the relevancy of such proof under Rule 401."); Giannelli, supra, at 1228-29.

The opposing view, espoused by Judge Weinstein, Professor Berger, and others, maintains that "[T]he silence of the rule and its drafters should be regarded as tantamount to an abandonment of the general acceptance standard." J. Weinstein & M. Berger, supra, p 702 at 702-16. See also C. Wright & K. Graham, 22 Federal Practice and Procedure Sec. 5168, at 86-90 (1978). Cf. State v. Williams, 388 A.2d 500, 503 (Me.1978) (interpreting Maine rules of evidence, which are patterned after the federal rules, as not incorporating Frye ). Arguing that Frye is inconsistent with the policies animating the Federal Rules of Evidence, this view focuses in particular on the broad scope of relevance in the federal rules. Fed.R.Evid. 401 specifically defines relevant evidence as "evidence having any tendency to make the existence of any fact that is of consequence to the determination of the action more probable or less probable than it would be without the evidence." Moreover, Fed.R.Evid. 402 provides that "[a]ll relevant evidence is admissible, except as otherwise provided by the Constitution of the United States, by Act of Congress, by these rules, or by other rules prescribed by the Supreme Court pursuant to statutory authority." Thus, because the mere relevance of novel scientific evidence does not hinge on its "general acceptance" in the scientific community, Rules 401 and 402, taken together, arguably create a admissibility of novel scientific evidence. Id. See standard of admissibility that is inconsistent with Frye. See C. Wright & K. Graham, 22 Federal Practice and Procedure Sec. 5168, at 89-90 (1978). Notwithstanding the appeal of this analysis, the notes of the advisory committee make clear that Rule 402 is limited by Fed.R.Evid. 403 and by the rules contained in Article VII of the Federal Rules of Evidence, including Rule 702. The touchstone of Rule 702, as was noted above, is the helpfulness of the expert testimony, i.e., whether it "will assist the trier of fact to understand the evidence or to determine a fact in issue." Fed.R.Evid. 702. Thus, the rules themselves contain a counterweight to a simple relevancy analysis.

Although we believe that "helpfulness" necessarily implies a quantum of reliability beyond that required to meet a standard of bare logical relevance, see discussion infra, it also seems clear to us that some scientific evidence can assist the trier of fact in reaching an accurate determination of facts in issue even though the principles underlying the evidence have not become "generally accepted" in the field to which they belong. Moreover, we can assume that the drafters of the Federal Rules of Evidence were aware that the Frye test was a judicial creation, and we find nothing in the language of the rules to suggest a disapproval of such interstitial judicial rulemaking. Therefore, although the codification of the rules of evidence may counsel in favor of a re-examination of the general acceptance standard, on balance we conclude that the Federal Rules of Evidence neither incorporate nor repudiate it. Cf. Saltzburg & Redden, supra, at 452 (suggesting the decision as to the proper standard for evaluating scientific evidence will continue to be made on a case-by-case basis). We will consider, therefore, the advantages of and the problems associated with Frye's general acceptance standard.

The most important justification for the Frye test is that it provides a method by which courts can assess the reliability of novel scientific expert testimony. See United States v. Addison, 498 F.2d 741, 743-44 (D.C.Cir.1974). The general acceptance standard in effect permits the experts who know most about a procedure to form a "technical jury," whose positive assessment of the scientific status of a procedure becomes a necessary prerequisite to the admissibility of expert testimony based on the procedure. See People v. Barbara, 400 Mich. 352, 405, 255 N.W.2d 171, 194 (1977). Adherents of the general acceptance standard also argue that it guarantees the existence of a coterie of experts qualified to testify about the status of a particular scientific technique and, in theory at least, promotes uniformity of decision. United States v. Addison, 498 F.2d at 744; Reed v. State, 283 Md. 374, 388, 391 A.2d 364, 371-72 (1978).

The general acceptance standard also safeguards against the possible prejudicial effects of testimony based upon "an unproved hypothesis in an isolated experiment." United States v. Brown, 557 F.2d 541, 556 (6th Cir.1977). The concern over potentially specious expert testimony assumes particular importance in the criminal context, where the general acceptance standard has had its most substantial impact.12 When the government seeks to introduce novel scientific evidence, for example, a possible tension between the defendant's right to a fair trial, on the one hand, and the trend toward admissibility of expert testimony embodied in the Federal Rules of Evidence, is apparent in many of the cases discussing also Giannelli, supra, at 1246 (introduction of unreliable evidence increases the likelihood of an erroneous verdict). Cf. id. at 1248 (suggesting that, in the criminal context, the defendant be allowed to introduce novel scientific evidence on the basis of a less stringent foundational showing than that required of the prosecution). This concern is also reflected by those courts that highlight the potentially prejudicial effect on the jury arising from the "aura of special reliability and trustworthiness" of scientific expert testimony. United States v. Amaral, 488 F.2d 1148, 1152 (9th Cir.1973). See also Addison, 498 F.2d at 744 (scientific evidence may "assume a posture of mythic infallibility in the eyes of a jury of laymen.").

Notwithstanding the valid evidentiary concerns subsumed in the general acceptance standard, critics of the standard have cited two general problems with it: its vagueness and its conservatism. See, e.g., 1 D. Louisell & C. Mueller, supra, Sec. 105, at 821. Professor Giannelli's excellent and comprehensive article catalogues the numerous difficulties that have arisen in applying the test. Giannelli, supra, at 1208-28. First, the vague terms included in the standard have allowed courts to manipulate the parameters of the relevant "scientific community" and the level of agreement needed for "general acceptance." Thus, some courts, when they wish to admit evidence, are able to limit the impact of Frye by narrowing the relevant scientific community to those experts who customarily employ the technique at issue. See, e.g., People v. Williams, 164 Cal. App. Supp. 2d 858, 331 P.2d 251 (App. Dep't Super.Ct. 1958) (in admitting results of the Nalline test for narcotics use, the court held that the Frye test was satisfied upon showing of general acceptance by those who are expected to be familiar with the use of the technique, although the prosecution's own expert had conceded the lack of acceptance within the medical profession generally).13 Judicial interpretation of the "general acceptance" component of the test has yielded even more disparate results. One court has described "general acceptance" as "widespread; prevalent; extensive though not universal," United States v. Zeiger, 350 F. Supp. 685, 688 (D.D.C.), rev'd 475 F.2d 1280 (D.C.Cir.1972), while another has suggested that the test requires agreement by a "substantial section of the scientific community." United States v. Williams, 443 F. Supp. 269, 273 (S.D.N.Y.1977), aff'd, 583 F.2d 1194 (2d Cir.1978), cert. denied, 493 U.S. 1117, 99 S. Ct. 1025, 59 L. Ed. 2d 77 (1979).

Professor Giannelli and others have discussed other problems that arise in applying the Frye test: the selectivity among courts in determining whether evidence derives from "novel" principles; the inadequacy of expert testimony on many scientific issues; an uncritical acceptance of prior judicial, rather than scientific, opinion as a basis for finding "general acceptance"; and the narrow scope of review by which some appellate courts review trial court rulings. See Giannelli, supra, at 1208-21. All of these problems contribute to the "essential vagueness" of the Frye test. 1 D. Louisell & C. Mueller, supra, at 821.

Apart from these various difficulties in implementation, moreover, Frye's general acceptance standard has been found to be unsatisfactory in other respects. Under Frye, some have argued, courts may be required to exclude much probative and reliable information from the jury's consideration, thereby unnecessarily impeding the truth-seeking function of litigation.14 See, e.g., United States v. Sample, 378 F. Supp. 44, 53 (E.D.Pa.1974) (Frye "precludes too much relevant evidence"); see also 1 D. Louisell & C. Mueller,supra, at 822; Lacey, "Scientific Evidence," 24 Jurimetrics, Journal of Law, Science, and Technology 254, 265 (1984) (Frye jurisdictions will always lag behind advances in science). But see United States v. Addison, 498 F.2d 741, 743 (D.C.Cir.1974) (consequence that Frye standard retards the admissibility of scientific evidence is not an "unwarranted cost").

In sum, the Frye test suffers from serious flaws. The test has proved to be too malleable to provide the method for orderly and uniform decision-making envisioned by some of its proponents. Moreover, in its pristine form the general acceptance standard reflects a conservative approach to the admissibility of scientific evidence that is at odds with the spirit, if not the precise language, of the Federal Rules of Evidence. For these reasons, we conclude that "general acceptance in the particular field to which [a scientific technique] belongs," Frye, 293 F. at 1014, should be rejected as an independent controlling standard of admissibility. Accordingly, we hold that a particular degree of acceptance of a scientific technique within the scientific community is neither a necessary nor a sufficient condition for admissibility; it is, however, one factor that a district court normally should consider in deciding whether to admit evidence based upon the technique.

The language of Fed.R.Evid. 702, the spirit of the Federal Rules of Evidence in general, and the experience with the Frye test suggest the appropriateness of a more flexible approach to the admissibility of novel scientific evidence. In our view, Rule 702 requires that a district court ruling upon the admission of (novel) scientific evidence, i.e., evidence whose scientific fundaments are not suitable candidates for judicial notice, conduct a preliminary inquiry focusing on (1) the soundness and reliability of the process or technique used in generating the evidence, (2) the possibility that admitting the evidence would overwhelm, confuse, or mislead the jury, and (3) the proffered connection between the scientific research or test result to be presented, and particular disputed factual issues in the case.15A.

In establishing the reliability of novel scientific evidence as one criterion of its admissibility under Rule 702, we join a growing number of courts that have focused on reliability as a critical element of admissibility. See, e.g., State v. Temple, 302 N.C. 1, 273 S.E.2d 273, 280 (1981) (evidence based on new scientific methods will be admitted when the demonstrated accuracy and reliability of the method have become established and recognized); State v. Kersting, 50 Or.App. 461, 623 P.2d 1095, 1101 (1981) (evidence based on a scientific technique that is not generally accepted may be admitted if there is "credible evidence on which the trial judge may make the initial determination that the technique is reasonably reliable"), aff'd, 292 Or. 350, 638 P.2d 1145 (1982); cf. D'Arc v. D'Arc, 157 N.J.Super. 553, 385 A.2d 278 (1978) (recognizing a distinction between the general acceptance standard and a standard based upon reliability, and upholding the admissibility of evidence that satisfies either test).16

The reliability inquiry that we envision is flexible and may turn on a number of considerations, in contrast to the process of scientific "nose-counting" that would appear to be compelled by a careful reading of Frye. Unlike the Frye standard, the reliability assessment does not require, although it does permit, explicit identification of a relevant scientific community and an express determination of a particular degree of acceptance within that community. The district court in assessing reliability may examine a variety of factors in addition to scientific acceptance. In many cases, however, the acceptance factor may well be decisive, or nearly so. Thus, we expect that a technique that satisfies the Frye test usually will be found to be reliable as well. On the other hand, a known technique which has been able to attract only minimal support within the community is likely to be found unreliable. See United States v. Williams, 583 F.2d 1194, 1198 (2d Cir.1978), cert. denied, 439 U.S. 1117, 99 S. Ct. 1025, 59 L. Ed. 2d 77 (1979).17

Where a form of scientific expertise has no established "track record" in litigation, the court may look to other factors that may bear on the reliability of the evidence.18 Judge Weinstein and Professor Berger have compiled a list of these factors, see 3 J. Weinstein & M. Berger, supra, p 702, at 702-18 to -19, some of which deserve explicit mention here. For instance, a court assessing reliability may consider the "novelty" of the new technique, that is, its relationship to more established modes of scientific analysis. This consideration may be especially significant in connection with an assessment of techniques that are new to the litigation setting. The existence of a specialized literature dealing with the technique is another factor. Both of these factors bear on the likelihood that the scientific basis of the new technique has been exposed to critical scientific scrutiny. See, e.g., State v. Bullard, 36 Crim.L.Rep. [BNA] 2173 (Dec. 5, 1984) (N.C.Sup.Ct. Nov. 6, 1984) (citing use of established procedures and independent research in admitting, under a reliability standard, footprint comparisons); State v. Hall, 297 N.W.2d 80 (Iowa 1980) (en banc) (novelty of the new invention and the existence of a specialized literature are relevant to the admissibility determination under Iowa version of Federal Rules of Evidence). The qualifications and professional stature of expert witnesses, and the non-judicial uses to which the scientific technique are put, may also constitute circumstantial evidence of the reliability of the technique. See 3 J. Weinstein & M. Berger, supra, at 702-19 nn. 10 & 11 (citing cases).

The frequency with which a technique leads to erroneous results will be another important component of reliability. At one extreme, a technique that yields correct results less often than it yields erroneous one is so unreliable that it is bound to be unhelpful to a finder of fact. Conversely, a very low rate of error strongly indicates a high degree of reliability. In addition to the rate of error, the court might examine the type of error generated by a technique.19 In a case involving the admission of voiceprint evidence, for example, the Second Circuit emphasized the fact that any shortcomings in the scientific technique or its application would result in the inability to match two voice spectrograms, rather than an erroneous conclusion that the two spectra were generated by the same voice. See Williams, 583 F.2d at 1198. Finally, the district may take judicial notice of expert testimony that has been offered in earlier cases to support or dispute the merits of a particular scientific procedure. Undoubtedly, other factors could be added to the list.

After assessing the reliability of the evidence, the court must also weigh any danger that the evidence might confuse or mislead the jury. It may seem paradoxical to suggest that scientific evidence based on principles bearing substantial indicia of reliability could confuse rather than assist the jury, but we do not doubt that this may be so, in some cases. One example might involve a technique which has "assume[d] a posture of mythic infallibility," Addison v. United States, 498 F.2d 741, 744 (D.C.Cir.1974), among lay persons, or at least one whose shortcomings are, for some reason, unlikely to be effectively communicated to the jury. The degree to which an unwarranted "aura of reliability," United States v. Baller, 519 F.2d 463, 466 (4th Cir.), cert. denied, 423 U.S. 1019, 96 S. Ct. 456, 46 L. Ed. 2d 391 (1975), attaches to scientific evidence will naturally vary with the type of evidence. The danger that scientific evidence will mislead the jury might be greater, for example, where the jury is not presented with the data on which the expert relies, but must instead accept the expert's assertions as to the accuracy of his conclusions. Cf. People v. Marx, 54 Cal. App. 3d 100, 111, 126 Cal. Rptr. 350, 356 (1975) (in support of its holding admitting bite mark comparison evidence, the court noted that "the basic data on which the experts based their conclusions were verifiable by the court"), cited in 3 J. Weinstein & M. Berger, supra, p 702 at 702-20 n. 18. Techniques that rely on the use of a mechanical device to produce data as well as upon the exercise of an expert's subjective judgment to draw conclusions from the data would also seem to raise at least the possibility of confusing or misleading the jury.20

The trial court must then balance its assessment of the reliability of a novel scientific technique against the danger that the evidence, even though reliable, might nonetheless confuse or mislead the finder of fact, and decide whether the evidence should be admitted. We decline to specify the foundational showing required for admissibility in traditional burden of proof terms, but see Giannelli, supra, at 1245-50 (advocating burden of proof test for determining the validity of novel scientific evidence), because the balancing analysis incorporates important policy elements (i.e., the likelihood that a particular type of evidence will mislead the jury) which render the determination something more than a fact-finding. Cf. McCormick, Scientific Evidence: Defining a New Approach to Admissibility, 67 Iowa L.Rev. 879, 908 (1982) (describing the admissibility issue with respect to novel scientific testimony as a "mixed question of law and fact"). Therefore, we will review district court decisions to admit or exclude novel scientific evidence by an abuse of discretion standard.21

We do, however, suggest some considerations that are germane to the balancing exercise. First, there is the "presumption of helpfulness" accorded expert testimony generally under Fed.R.Evid. 702. In re Japanese Electronics Products Antitrust Litigation, 723 F.2d at 279. See also 3 J. Weinstein & M. Berger,supra, p 702 at 702-21 ("the relevancy approach favors admissibility whenever the general conditions for admissibility of evidence have been met"). We also note, however, that some caution is appropriate, especially in the criminal context, whenever proffered novel scientific evidence, if unreliable or likely to mislead, will increase the likelihood of an erroneous verdict. The trial court, in ruling upon the admissibility of this kind of evidence, should take care to ensure that any weaknesses in the proffered evidence will be fully explored before or during the trial.22 The extent to which the adverse party has had notice of the evidence and an opportunity to conduct its own tests or produce opposing experts are also appropriate considerations for the court. See United States v. Baller, 519 F.2d at 466.

With respect to the procedure that district courts should follow in making preliminary determinations regarding admissibility of evidence, we recognize that " 'the control of the order of proof at trial is a matter committed to the discretion of the trial judge.' " United States v. Ammar, 714 F.2d 238, 246 (3d Cir.1983) (quoting United States v. Continental Group, Inc., 603 F.2d 444, 456 (3d Cir.1979), cert. denied, 444 U.S. 1032, 100 S. Ct. 703, 62 L. Ed. 2d 668 (1980)), cert. denied sub nom. Stillman v. United States, --- U.S. ----, 104 S. Ct. 344, 78 L. Ed. 2d 311 (1983). Hence, we will not prescribe any mandatory procedures that district courts must follow in every case involving proffers of scientific evidence. A few general observations in this regard are appropriate, however.

It would appear that the most efficient procedure that the district court can use in making the reliability determination is an in limine hearing. Such a hearing need not unduly burden the trial courts; in many cases, it will be only a brief foundational hearing either before trial or at trial but out of the hearing of the jury. In the course of the in limine proceeding, the trial court may consider, inter alia, offers of proof, affidavits, stipulations, or learned treatises, see Fed.R.Evid. 803(18), in addition to testimonial or other documentary evidence (and, of course, legal argument). In addition, the court may properly consider the testimony presented to other courts that have addressed the same evidentiary issue, and the opinions of those courts on the subject. If a technique has found favor with a significant number of other courts, a district court may exercise its discretion to admit the evidence through judicial notice.

Having generally set out the appropriate inquiry, we now turn to the facts of this case. Unfortunately the district court never addressed the reliability question because it essentially--and erroneously--concluded that expert evidence of this type could never assist the trier of fact. From the facts available on the record and otherwise, it would appear that the scientific basis for the expert evidence in question is sufficiently reliable to satisfy Rule 702. In a recent case approving the use of expert testimony on eyewitness perception and memory in certain circumstances, the California Supreme Court noted the proliferation of empirical research demonstrating the pitfalls of eyewitness identification and concluded that "the consistency of the results of these studies is impressive." People v. McDonald, 37 Cal. 3d 351, 208 Cal.Rptr. at 245, 690 P.2d at 718 (1984).23 The government has not had an opportunity to make a presentation on this point, however, and we believe that the district court should make the determination in the first instance.

An additional consideration under Rule 702--and another aspect of relevancy--is whether expert testimony proffered in the case is sufficiently tied to the facts of the case that it will aid the jury in resolving a factual dispute. In this regard, we hold that a defendant who seeks the admission of expert testimony must make an on-the-record detailed proffer to the court, including an explanation of precisely how the expert's testimony is relevant to the eyewitness identifications under consideration. The offer of proof should establish the presence of factors (e.g., stress, or differences in race or age as between the eyewitness and the defendant) which have been found by researchers to impair the accuracy of eyewitness identifications. See People v. McDonald, supra, 37 Cal. 3d 351, 208 Cal.Rptr. at 253, 690 P.2d at 726. See discussion of Chapple, supra text preceding note 7, in which the proffered evidence was found to be sufficiently tied to the case. Failure to make such a detailed proffer is sufficient grounds to exclude the expert's testimony. See United States v. Fosher, 590 F.2d 381, 383 (1st Cir.1979).24

Turning to the facts of the present case, we note that appellant made no such on-the-record proffer, but conclude that this defect is not fatal on this appeal for two reasons. First, the district court did not rely on this failure to justify its decision to exclude appellant's expert. Second, the district court conducted all proceedings concerning the admissibility of appellant's expert's testimony off the record. With the resulting undeveloped record, we simply have no way of judging whether the proffered testimony was sufficiently tied to the facts of the case. The encounters between the identification witnesses and appellant were apparently not under conditions of stress, nor so far as we know, was there a potential cross-racial identification problem. It is conceivable, however, that the off-the-record expert proffer made by the appellant was sufficiently tied to the facts of the case to satisfy Rule 702. Therefore we cannot affirm the district court's exclusion of the expert evidence on this ground, and the district court will have to explore this question of "fit" on remand.

The Rule 702 analysis we have outlined incorporates to some extent a consideration of the dangers, particularly the danger of unfair prejudice, enumerated in Fed.R.Evid. 403. On remand, however, even if the proffered evidence satisfies Rule 702, the district court may decide nonetheless to invoke Rule 403 to exclude the evidence if the court finds its probative value to be substantially outweighed by other dangers, e.g., confusion of the issue or waste of time. Fed.R.Evid. 403. The availability of Rule 403 is especially significant when there is evidence of a defendant's guilt other than eyewitness evidence, e.g., fingerprints, or other physical evidence. There is no such evidence in this case, however. See supra section II.

The availability of other methods that would serve the purposes for which the appellant seeks to introduce expert testimony may also serve to justify exclusion under Rule 403, though we do not perceive the viability of such an alternative in this case. Moreover, as an abstract proposition, it would seem anomalous to hold that the probative value of expert opinion offered to show the unreliability of eyewitness testimony so wastes time or confuses the issue that it cannot be considered even when its putative effect is to vitiate the only (eyewitness) evidence offered by the government.25 We understand the government to argue, however, that, because the eyewitness encounters in this case were more than brief in duration and because there are so many eyewitnesses who identified appellant, the offered expert testimony would be of no utility. In exceptional circumstances this, too, might concededly constitute a Rule 403 balancing factor.26 In any event, because the district court did not mention Rule 403 on the record or conduct the balancing required by that rule, see United States v. Long, 574 F.2d 761, 770 (3d Cir.1978) (Adams, J., concurring), we decline to decide the Rule 403 question at this stage of the litigation.27

For the reasons set forth above, we must vacate the judgment of conviction entered against the appellant. As we have explained, because the crucial evidence against appellant consisted solely of eyewitness identification, we cannot conclude on this record that the district court's error in excluding the proffered expert testimony is harmless. See supra note 26. Appellant was deprived at most, however, of the opportunity to present his expert witness. The district court's error will become harmless if on remand the district court, in the exercise of its Rule 702 or 403 discretion, decides that the proffered testimony is not admissible. Therefore, we conclude that a new trial is required only if the district court determines that the proffered testimony is admissible. Cf. Waller v. Georgia, --- U.S. ----, 104 S. Ct. 2210, 2217, 81 L. Ed. 2d 31 (1984) ("If, after a new suppression hearing, essentially the same evidence is suppressed, a new trial presumably would be a windfall for the defendant and not in the public interest"); United States v. Faison, 679 F.2d 292, 297-98 (3d Cir.1982).

Therefore, the judgment of conviction will be vacated and the case will be remanded to the district court for an evidentiary hearing concerning the admissibility of appellant's proffered expert testimony. If the court determines that the expert testimony should have been admitted, it is directed to grant a new trial. If the court decides that the testimony is not admissible under Rule 702 (or should be excluded under Rule 403), then the judgment of conviction against appellant should be reinstated.28

DUMBAULD, Senior District Judge, concurring.

Rule 403 of the Federal Rules of Evidence provides that "although relevant, evidence may be excluded if its probative value is substantially outweighed ... by considerations of undue delay, waste of time, or needless presentation of cumulative evidence."

In excluding defendant's proffer of an expert witness regarding the unreliability of eyewitness identification testimony (a weakness known at least since Hugo Munsterberg's experiments at Harvard and ordinarily presented adequately to the jury by argument of counsel), the District Judge did not invoke Rule 403 eo numero or place on the record any "extensively articulated ... extensive explication" thereunder.¹

However, under part V of Judge Becker's opinion (which is truly a minor magnum opus of jurisprudential virtuosity) the District Court upon remand remains free to do so without preclusion by our present decision of its exercise of discretion under Rule 403. I therefore concur in the disposition made of the case.

I agree with the Court's opinion that there can be cases where expert opinion of this type may be useful. But with respect to the case at bar it seems plain to me that any error by the District Judge was harmless. A dozen witnesses, who had spent from 5 to 45 minutes in negotiations with the defendant while he "conned" them with his fraudulent scheme, identified him. This case did not involve a momentary glimpse of a bank robber at the teller's window or a rape perpetrated under a ski mask. It would be unfortunate if his conviction and the time spent at his trial were to go down the drain because of an academic error regarding the intellectual foundations for judicial acceptance of novel scientific disciplines.