Error and the Growth of Experimental Knowledge

Philosophers of Science attempt to construct a normative methodology for scientific inference. As a practicing scientist who follows this field, my impression is that the majority of the published work in this area falls into one of three categories: Critiquing existing methodologies, improving on existing methodologies, and less frequently, proposing a new methodology that is felt to have solved the philosophical problems of the previous methods. Dr. Mayo's book falls into the third category. In the last half of this century, the debate in this field has centered around the "Falsification" methodology of Karl Popper. In testing deterministic hypotheses, Popper observed that hypotheses can be proven wrong with falsifying evidence by applying the logical rule of modus tollens. However, when experimental evidence is consistent with the hypothesis, believing that this confirms the hypothesis is committing the "affirming the consequent" fallacy. Hence, scientific "theories" cannot be verified, but only falsified. Subsequent philosophers pointed out a number of shortcomings with this method: it cannot be applied to statistical hypotheses (which is the area were methodology is most important to practicing scientists), the Duhem problem- an observation can only falsify a hypothesis if it is conclusively certain, and the problem of auxiliary assumptions- when data is inconsistent with the hypothesis, the test is not informative regarding whether the main hypothesis is false or whether the problem resides in the ever-present auxiliary assumptions that are necessary to connect the hypothesis with experimental implications. Subsequent philosophers addressed these methodological faults by applying the principles of Bayesian statistics to the problem of testing hypotheses; using experimental evidence to transform an a priori probability of a hypothesis being valid to an a posteriori probability. Ideas from a number of Bayesian philosophers were synthesized by Howson and Urbach in their excellent book, Scientific Reasoning: The Bayesian Approach. This method applies equally well to both statistical and deterministic hypotheses. Also, the Bayesians claim neat solutions to other problems with Popper, e.g., auxiliary assumptions and Duhem. Predictably, other philosophers were quick to find faults with this approach. The most common objections are: the necessity of subjective prior probabilities and the "Problem of Old Evidence." Mayo, in her book Error and the Growth of Experimental Knowledge, presents an Error-Statistical approach to scientific inference. She confronts head-on three important issues that any normative method must address: What counts as experimental evidence?, How to assess the acceptability of auxiliary assumptions and how to rule out alternative hypotheses?, and How to falsify statistical hypotheses? The first is addressed by explicitly identifying and justifying assumptions of the experimental data. The second issue