by Kenneth R. Foster and Peter W. Huber
What is scientific knowledge, and when is it reliable? These deceptively simple questions have been sources of endless controversy. Whether Creation Science should be taught in schools along with the theory of evolution turns on whether Creation Science -- or evolution, for that matter -- can fairly be called science rather than belief, faith, or something else. In the courtroom, the outcomes of criminal, paternity, first amendment, and civil liability cases (among others) often turn on scientific evidence, the reliability of which may be hotly contested. Scientists have been arguing for years about the risks or nonrisks of radon in the home, stilbestrol residues in food,1 and other potential subtle causes of injury. Major policy disputes revolve around these issues. Sometimes, as with DNA testing in capital murder trials, the reliability of a claim presented as scientific is a matter of life and death.
In 1993, the U.S. Supreme Court handed down a landmark ruling on scientific evidence, Daubert v. Merrell Dow Pharmaceuticals.2 "Faced with a proffer of expert scientific testimony," Justice Blackmun wrote for a seven-Justice majority, "the trial judge must determine . . . whether the expert is proposing to testify to (1) scientific knowledge that (2) will assist the trier of fact to understand or determine a fact in issue. . . . Many factors will bear on the inquiry, and we do not presume to set out a definitive checklist or test. But some general observations are appropriate."3 The Court's "general observations" followed.
Chief Justice Rehnquist, joined by Justice Stevens, dissented. In a passage that says much about the Daubert controversy and about the purpose of this book, the Chief Justice wrote:
"General observations" by this Court customarily carry great weight with lower federal courts, but the ones offered here suffer from the flaw common to most such observations -- they are not applied to deciding whether or not particular testimony was or was not admissible, and therefore they tend to be not only general, but vague and abstract. . . . Twenty-two amicus briefs have been filed in the case, and indeed the Court's opinion contains no less than 37 citations to amicus briefs and other secondary sources. The various briefs filed in this case are markedly different from typical briefs . . . they deal with definitions of scientific knowledge, scientific method, scientific validity, and peer review -- in short, matters far afield from the expertise of judges. . . .4
The Daubert majority did rely on some unusual authorities. Many of its references covered familiar legal territory -- articles in law reviews, legal treatises, and advisory notes to the Federal Rules of Evidence.5 But the Court went much further. It cited two of the twentieth century's most influential philosophers of science, Carl Gustav Hempel and Karl Raimund Popper, as well as John Ziman, a prominent physicist turned commentator on science. The Court cited the editors of influential medical journals. It also cited three amicus briefs filed on behalf of groups of scientists, quoting from two of them.6 One of those groups comprised eighteen scientists, including six Nobel laureates,7 with expertise in chemistry, physics, meteorology, epidemiology, environmental medicine, and teratology (the study of malformation). The second scientists' amicus brief had been filed on behalf of the American Association for the Advancement of Science and the National Academy of Sciences. A third group of scientists that filed with the Court included Stephen Jay Gould, a well-known author and paleontologist.
As Chief Justice Rehnquist noted in his dissent, Supreme Court opinions do not ordinarily rest on this kind of intellectual foundation. Few of the scientific sources cited by the majority would be readily at hand for most judges to consult, nor would the broader literature that those sources summarize and represent. Yet seven of the Justices agreed that the meaning of a key phrase in the Federal Rules of Evidence -- "scientific knowledge" -- cannot be given intelligent meaning without venturing beyond the standard law library into the domains of science and philosophy. . . .
. . .The book is not a legal treatise; we do not dwell on the large law-review literature that addresses Frye or on the technical legal issues related to admissibility of specific types of scientific evidence.41 Nor is this a philosophical treatise; we do not spend much time on Popper's technical philosophy (on falsification as a demarcation criterion of science) or on Thomas Kuhn's (on the scientific community). This book is, instead, an extended commentary on "scientific validity" and the law's rules of evidence. We attempt to explain the significance of the Daubert criteria in lay terms, providing key references to philosophical, scientific, and other sources and focusing on sources accessible to lay readers.
Our purpose is to explore a concrete, practical question: When should evidence that is framed as scientific be considered reliable enough to be presented to a jury? The second question we address is much broader: What is necessary or sufficient to establish "scientific validity" for more general purposes in scientific, regulatory, and other communities that use scientific evidence to make legal, social, and political decisions?
We discuss the Daubert criteria and how issues of scientific validity play out in the scientific and legal arenas. We also include brief selections from papers, most of them written by prominent scientists and philosophers, that bear on the issues of scientific validity and inference. These papers were not written as prescriptions for judges or lawyers; they are discussions from outside the legal community that bear on the terms, ideas, and standards alluded to in Daubert.
Judges plainly cannot surrender to scientists their responsibilities as gatekeepers of evidence, nor can they insist on impossibly high standards of scientific rigor. The general criteria outlined in Daubert are, however, similar to those that scientists use to evaluate scientific evidence. They are also similar to the criteria that any intelligent layperson would use to evaluate empirical claims about the world.
To illustrate the issues, we include excerpts of testimony proffered in Daubert v. Merrell Dow Pharmaceuticals42 and other Bendectin cases. We also include a brief summary of some of the major scientific developments relative to the reproductive toxicity of Bendectin. We do not intend to re-argue Daubert or to offer unsolicited advice to the judges, lawyers, and juries who are still working on Bendectin cases. We use Bendectin simply to supply a concrete example of the much larger issues at stake in Daubert, and of the still larger issues at stake in every debate about scientific knowledge.
This book is organized around the criteria set out in Daubert. In chapter 2 we consider the issue of "fit" -- whether a plausible theory relates specific facts to the larger factual issues in contention. In chapter 3 we discuss some of the philosophical questions that the Supreme Court raised in Daubert, particularly the concept of "falsification" of scientific claims. In chapter 4 we discuss the (sometimes spectacular) errors that scientists can make. In chapter 5 we consider the issue of "reliability" in science and address what makes evidence reliable or unreliable in fields such as epidemiology and toxicology. In chapter 6 we explore the meaning of "scientific validity." In chapter 7 we address peer review and the problem of setting boundaries. In chapter 8 we consider a group of problems that bear on the hazards of confusion and prejudice in presenting science to a jury. In the concluding chapter we attempt to reconcile the law's need for workable rules of evidence with the views of scientific validity and reliability that emerge from scientific and other disciplines.
But we leave prescription to the last. As the psychologist Jonathan Baron points out in his book Thinking and Deciding, descriptive theory tells us how people "do" think when making decisions in the face of uncertainty.43 Prescriptive theory tells us how they should think. Normative theory tries to identify rules of good thinking, to meet the needs of the thinker. Most of this book addresses the "do" -- the ways that scientists go about their business, the ways in which they do or do not converge on answers, the ways in which they make mistakes, correct them, and pursue "reliability," "validity," and the scientific conception of "truth."
The goal of science is epistemic -- to achieve "cosmic understanding," as the Daubert majority put it.44 The goal of the judicial process is "the particularized resolution of legal disputes."45 The task of a judge faced with questionable scientific testimony has little to do with questions of cosmic truth; the task is to apply the Federal Rules of Evidence evenhandedly, in a way that is faithful to the language of the rules and that makes sense in the very practical context of passing specific judgment on a specific claim. But these rules, as interpreted in Daubert, involve criteria that must derive, somehow, from science itself.
Judges can -- and will -- take their best shot at an answer. That best shot will get better as lawyers and judges gain a better understanding of how science works. An old formula holds that science is close reasoning pushed up against close observation. At that level, science is not so different from the law after all.46
1. A. I. Marcus, Cancer from Beef (Johns Hopkins University Press, 1994); L. Cole, The Politics of Radon (AAAS Press, 1994). See also Phantom Risk, ed. K. Foster et al. (MIT Press, 1993), p. 101.
2. Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993). The majority opinion and the dissenting opinion are reproduced in their entirety in appendix B below.
3. Daubert, 509 U.S. at 592-593.
4. Ibid. at 598-599.
5. E. Green and C. Nesson, Problems, Cases, and Materials on Evidence (Little, Brown, 1983), p. 649; Webster's Third New International Dictionary (1986), p. 1252 ('scientific'); B. Black, A unified theory of scientific evidence, Fordham Law Review 56 (1988): 595-599; J. Starrs, Frye v. United States restructured and revitalized: A proposal to amend Federal Evidence Rule 702, Jurimetrics J. 26 (1986): 249-256; Advisory Committee's Notes on Fed. Rule Evid. 602; Advisory Committee's Notes on Art. VIII of the Rules of Evidence; 3 J. Weinstein & M. Berger, Weinstein's Evidence P 702[03], pp. 702-718 (1988); ibid., P 702[03], pp. 702-741 to 702-742; S. Jasanoff, The Fifth Branch: Science Advisors as Policymakers (Harvard University Press, 1990), pp. 61-76; M. McCormick, Scientific evidence: Defining a new approach to admissibility, Iowa Law Review 67 (1982): 879, 911-912; Symposium on science and the Rules of Evidence, 99 F.R.D. 187, 231 (1983) (statement by Margaret Berger); J. Weinstein, Rule 702 of the Federal Rules of Evidence is sound; It should not be amended, Federal Rules Decisions 138 (1991): 631-632; B. Cardozo, The Nature of the Judicial Process (Yale University Press, 1921), pp. 178-179.
6. Daubert, 509 U.S. at 590 (citing and quoting from Brief for Nicolaas Bloembergen et al. as amici curiae 9); ibid. (citing and quoting from Brief for American Association for the Advancement of Science and the National Academy of Sciences as amici curiae 7-8); ibid. at 2798 (citing Brief for Ronald Bayer et al. as amici curiae). 7. Nicolaas Bloembergen (physics, 1981), Dudley Herschbach (chemistry, 1986), Jerome Karle (chemistry, 1980), Wassily Leontief (economics, 1973), William Lipscomb (chemistry, 1976, and Arno Penzias (physics, 1978). . . .
. . .41. The Federal Judicial Center's recent Reference Manual on Scientific Evidence (West, 1994) serves the latter purpose quite well.
42. 727 F. Supp. 570 (S.D. Cal. 1989), aff'd, 951 F.2d 1128 (9th Cir. 1991), vacated and remanded, 509 U.S. 579 (1993), aff'd, 43 F. 3d 1311 (9th Cir. 1995), cert. denied, 116 S. Ct. 189 (1995).
43. J. Baron, Thinking and Deciding, second edition (Cambridge University Press, 1994).
44. Daubert, 509 U.S. at 597.
45. Ibid.
46. See H. Margolis, Paradigms and Barriers: How Habits of the Mind Govern Scientific Beliefs (University of Chicago Press, 1993), p. 178.