Evidence based medicine: philosophical

Philosophy

BioMed CentralBMC Health Services Research ssOpen AcceDebate A philosophical analysis of the evidence-based medicine debate Scott R Sehon*1 and Donald E Stanley2 Address: 1Department of Philosophy, 8400 College Station, Bowdoin College, Brunswick, ME 04011, USA and 2Associates in Pathology, 500 West Neck Road, Nobelboro, ME 04555, USA Email: Scott R Sehon* email@example.com; Donald E Stanley email@example.com * Corresponding author Abstract Background: The term "evidence-based medicine" (or EBM) was introduced about ten years ago, and there has been considerable debate about the value of EBM. However, this debate has sometimes been obscured by a lack of conceptual clarity concerning the nature and status of EBM. Discussion: First, we note that EBM proponents have obscured the current debate by defining EBM in an overly broad, indeed almost vacuous, manner; we offer a clearer account of EBM and its relation to the alternative approaches to medicine. Second, while EBM proponents commonly cite the philosophical work of Thomas Kuhn and claim that EBM is a Kuhnian 'paradigm shift,' we argue that such claims are seriously mistaken and unduly polarize the EBM debate. Third, we suggest that it is much more fruitful to understand the relationship between EBM and its alternatives in light of a different philosophical metaphor: W.V. Quine's metaphor of the web of belief. Seen in this way, we argue that EBM is an approach to medical practice that is indeed importantly different from the alternatives. Summary: We can have a more productive debate about the value of EBM by being clearer about the nature of EBM and its relationship to alternative approaches to medicine. Background Over the past decade, the term "evidence-based medicine" (or EBM) has gained considerable currency. EBM has been described as a "paradigm shift" [1] that will "change medical practice in the years ahead."[2] Some suggest that the principles of EBM should be part of the standard training of all physicians and that those physicians who violate its precepts should ultimately face license suspension.[3] Critics of EBM claim that "there is no evidence (and unlikely ever to be) that EBM provides better medical care," and that EBM is simply "following its own political agenda."[4] Other critics use even harsher rhetoric, claiming that EBM's assumptions are "absurd" and "irrational".[5] Some commentators claim a middle ground by saying that EBM and other approaches should be "harmonized".[6] The critics argue that EBM proponents overemphasize the value of clinical trials. These critics suggest that there are other aspects to medicine. Some note the fundamental role of basic science in understanding the physiologic mechanisms of the body, the biology of disease, and the cellular targets of drugs.[5,7] There are also critics who emphasize the value of clinical experience and the judgment of individual physicians; [7–10] these critics sometimes emphasize the art of medicine, [10] and contrast this with the science of medicine, or they speak of techne vs. theory or compassion vs. reason. And other critics might argue that there are circumstances in which obserPublished: 21 July 2003 BMC Health Services Research 2003, 3:14 Received: 01 April 2003 Accepted: 21 July 2003 This article is available from: http://www.biomedcentral.com/1472-6963/3/14 © 2003 Sehon and Stanley; licensee BioMed Central Ltd. This is an Open Access article: verbatim copying and redistribution of this article are permitted in all media for any purpose, provided this notice is preserved along with the article's original URL.Page 1 of 10 (page number not for citation purposes) BMC Health Services Research 2003, 3 http://www.biomedcentral.com/1472-6963/3/14vational studies (or outcome studies)are a better choice than clinical trials [11]. The questions raised by this debate are fundamental: How should physicians practice medicine? Should they do so in accordance with the principles of EBM? If so, how should this be accomplished? How should health care dollars best be spent? While there has been considerable debate about these questions, it seems to us that the debate also involves some confusion. The confusion is a good indication that some conceptual clarification is in order, and that is our principal aim in this paper. Our philosophical analysis will be limited to conceptual and epistemological issues; there are ethical issues that arise in the practice of evidence-based medicine, but we will not be addressing those questions. Philosophers distinguish different types of questions. The questions just raised are, in this context, the basic questions, or the first order questions. By contrast, second order questions are questions about the first order questions, e.g., questions about the concepts employed in the first order questions. We can also refer to the second order questions as conceptual questions. In the present context, the second order questions would include: what is EBM? What are the alternatives to EBM? What is the relationship between EBM and the alternative approaches to medicine? Does EBM represent a paradigm shift? Without clear and compelling answers to these second order questions, the debate about the first order questions will be an exercise in futility. Obviously, if both sides in the debate have different ideas about what EBM is, then the opposing sides will be talking past each other when they argue about whether or not EBM should be practiced. We have to agree on what we are talking about before we can make progress on the question of whether it is a good thing. This seems obvious, but in the first section of our discussion we will argue that some overly broad characterizations of EBM have in fact done much to obscure the debate. In section two, we will argue that it is unhelpful and misleading to portray EBM as a paradigm shift and that such portrayals have polarized the debate in an unfortunate way. We will claim that it is much more useful to see EBM and its alternatives in light of a different philosophical picture: W.V. Quine's web of belief metaphor.[12] Seen in this light, we can see that EBM is an approach to medical practice that is indeed importantly different from the alternatives. The Quinean metaphor helps us to understand these differences, and this in turn puts us in a position to evaluate much more effectively the first order normative questions about how medicine ought to be practiced. We will begin in the third section by explaining certain aspects of his view that are most relevant to the EBM debate before applying the view in the fourth and final section. Discussion What is EBM? As has been noticed by others [7,13], the debate about the value of EBM has been muddied by an unfortunate tendency to define the term "evidence-based medicine" in an overly broad manner. For example, here is one definition of EBM that has some claim to being authoritative: "The conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients" [14]. In a similar vein, Brian Haynes writes "Advocates of evidence-based medicine want clinicians and consumers to pay attention to the best findings from health care research that are both valid and ready for clinical application".[15] Taken at face value, these definitions seem merely to say that EBM is the wise use of the best evidence available. Given that characterization alone, it would be astonishing that there is any dispute about EBM. It would be equally astonishing that anyone could think EBM, defined in this manner, revolutionary or even useful. After all, who could possibly be opposed to using the best evidence wisely? The very term "evidence-based medicine" seems similarly vacuous – as if any alternative to EBM means doing medicine based on something other than evidence. Even homeopaths base their treatments on evidence of a sort, and we suspect that homeopaths would even claim that they are conscientiously and explicitly using the best evidence available. Of course they are wrong about the latter claim, but nothing in these standard definitions of EBM tells us why homeopathy fails to be evidence-based medicine. Furthermore, few would suggest that the very idea of using evidence in medicine constitutes a revolutionary paradigm shift. Perhaps advocates of the term "evidencebased medicine" have a philosophical theory of evidence, a theory according to which everything that has passed for evidence prior to the EBM movement was not evidence at all. However, if they have such a theory, we have not seen it expressed. In the Background section, we mentioned a number of criticisms that have been raised against EBM. With such critics, we have the beginnings of an intelligible debate, for we have several reasonably distinct approaches to intervention questions, and we can ask substantive questions about the strengths and weaknesses of each, and we can ask which should be used when or how they should be combined. The present point is simply this: there is a tendency for some proponents of EBM to duck these questions and avoid this debate by defining "evidence-based medicine" such that it includes the best possible combination of basic science, clinical experience, and clinical tri-Page 2 of 10 (page number not for citation purposes) BMC Health Services Research 2003, 3 http://www.biomedcentral.com/1472-6963/3/14als. In so doing, the proponents of EBM come awfully close to simply defining EBM as the best way to practice medicine, whatever that may be. In other words, they respond to the second order conceptual question ("what is EBM?") by saying that EBM is whatever approach to medicine best answers the first order normative question ("how ought we to practice medicine?"). The proponents of EBM thereby give the illusion of having answered both sets of questions when in fact they have answered neither. They thus walk away from an important and substantive debate. We emphasize that our concern about the definition of EBM stems not merely from an academic desire for philosophical clarity. Our concern is eminently practical: we believe that the debate between EBM and alternative approaches is important and can change medical practice. But if EBM is explicitly defined in a way that is essentially vacuous, the debate will be unproductive. If this debate is to yield fruit, we must therefore clarify what we mean when we speak of EBM. The heart of EBM is the use of randomized controlled trials (RCTs) to determine the safety and efficacy of interventions, e.g. treatment, diagnosis, and prevention. Of course, EBM advocates have taken care to allow a role for other factors beyond evidence from RCTs. In particular, one must take into account the clinical state and circumstances of the patient, as well as the patient's own preferences and actions.[14,16–18] Even for EBM practitioners, the research evidence is but one component of medical decision making. But of course alternative approaches to medical practice will also take into account the patient's clinical state and values; this is not what separates EBM from the other approaches. What separates EBM from other approaches is the priority it gives to certain forms of evidence, and according to EBM the most highly prized form of evidence comes from RCTs (including systematic reviews) and meta-analyses of RCTs. So, for the purposes of this paper, we will take the term "evidence-based medicine" to refer essentially to the practice of taking RCTs as the strongly preferred form of medical evidence. Is EBM a paradigm shift? Proponents have hailed EBM as a "paradigm shift" or "revolution", in the sense of those words put forward by historian of science Thomas Kuhn.[19] At the same time, Brian Haynes acknowledges that "most scientists and EBM advocates are ignorant of the philosophy of science and give little or no thought to constructing a philosophical basis for their activities."[15] Indeed, Haynes goes on to write that "One hopes that the attention of philosophers will be drawn to these questions" [15]. In that spirit, we want to examine more closely the claim that EBM is a Kuhnian paradigm shift. The Kuhnian metaphor is familiar to philosophers, but its implications are presumably less familiar to physicians. We have nothing against a metaphorical characterization of EBM and its alternatives; well-chosen metaphors can do much to illuminate conceptual (i.e., second-order) questions, and thereby can help us answer the normative (i.e., first-order) questions. However, we will suggest that the Kuhnian metaphor is a poor one and serves to obscure more than it illuminates. According to Kuhn, "normal science" takes place when the scientific community shares what Kuhn calls a "paradigm". A paradigm is "entire constellation of beliefs, values, techniques, and so on shared by the members of a given community"[19]. The paradigm represents the shared context, and within that context, the scientists will work on remaining puzzles or problems. But within normal science, scientists do not seek "major substantive novelties"; [19] rather, "Mopping-up operations are what engage most scientists throughout their careers".[19] Kuhn suggests that the history of science is mostly constituted by periods of normal science. However, sometimes an accepted paradigm undergoes a crisis, followed by a 'scientific revolution' which overthrows the old paradigm and replaces it with a new paradigm. With the new paradigm, the scientific community has a new set of assumptions and a new set of puzzles to be solved. Kuhn refers to these revolutions as "changes of world view"; he suggests that "after a revolution scientists are responding to a different world".[19] Somewhat less metaphorically, Kuhn says that the post-revolutionary paradigm is, at least to some extent, incommensurable with the previous paradigm. Is EBM a paradigm shift? To answer this question, we have to specify the alternative with which we are comparing EBM. One alternative to EBM is the basic science approach: studying the physiological mechanisms of the body and the biochemical properties of drugs. We think that EBM is clearly not a paradigm shift in relation to this. As we will explain more fully in section four, EBM is so clearly intertwined with and complementary to basic science that it would make little sense to see EBM as a paradigm shift away from basic science. On the other hand, when we compare EBM with clinical experience and observational studies, we do have a shift in the sort of evidence that is most highly valued for diagnosis, therapy, and prognosis questions. Clinical experience and observational studies have much in common with each other and with RCTs. In all three cases, physicians attempt to answer the question of whether intervention X is effective for condition Y, and they do so by trying X in a certain number of cases and see whether it works. If a physician relies on her clinical experience, she reflects on past cases that were similar in certain respects and in which intervention X was tried. Observational studies are anPage 3 of 10 (page number not for citation purposes) BMC Health Services Research 2003, 3 http://www.biomedcentral.com/1472-6963/3/14attempt to make this more systematic: a large number of patients with Y are given intervention X, and the results are observed and summarized. This improves upon unsystematic clinical experience because it avoids some problems of bias or selective memory. RCTs take clinical experience several steps further by imposing fairly elaborate controls. So, the move to RCTs certainly constitutes a shift in some of the techniques of medical researchers. The move also reflects a change in some beliefs as well, for it reflects a change of view about the reliability of uncontrolled studies or clinical experience. So in moving from clinical experience to RCTs, we do have a change in many of the elements which make up a paradigm, according to Kuhn. Is the change enough to count as a paradigm shift? That is difficult to say, and we don't think the difficulty can be resolved by a more detailed description of the changes in medical research brought about by RCTs. The difficulty lies in the fundamental unclarity of the Kuhnian notion of a paradigm. The question is this: at what point do changes in theory and practice of science amount to a paradigm shift? Where is the dividing line between changing one's beliefs and practices within a paradigm and moving to a new paradigm? If we do not have any idea what sort of dividing line there might be, then it is not clear that talk of paradigms comes to anything at all. The move to RCTs changed considerably the specific techniques behind studies of treatment effectiveness. On the other hand, there was no immense change in doctrines. Instead, there was a recognition of the fact that evidence gathered through clinical experience and observational studies was subject to a number of flaws: expectation effects (on the part of both the patients and the physicians), the fact that most medical conditions improve regardless of treatment, etc. So was this sufficient to count as a paradigm shift? Absent a clearer dividing line or a clear account of the point of calling some advances paradigm shifts, we suggest that any answer to this question would be arbitrary and the exercise pointless. On behalf of Kuhn, one might suggest that a scientific community employs a revolutionary new paradigm precisely when the new theory and practice are incommensurable with the old. However, this suggestion creates two problems: one for the proponent of EBM specifically; and one for Kuhn's account more generally. The first problem is this: it is simply not credible to think that the theory and practices introduced by EBM are incommensurable with the previous practices and theory. Two theories would be incommensurable if they are untranslatable – i.e., if there are significant statements within each theory that cannot be translated at all into the language of the other theory. Thus neither theory could be completely understood using only the terms and concepts of the language of the other theory. Accordingly, incommensurability suggests that scientists working under distinct paradigms would not be able to comprehend one another's claims. However, by this standard, it seems clear that there is no incommensurability between medical research as practiced before EBM and medical research after the emphasis on RCTs. Physicians working under each of the supposedly different paradigms would have no trouble understanding one another's claims. Any number of extant texts [14,20,21] describe the theory and practice of EBM, and the description is couched in terms that would be fully comprehensible to a medical researcher in the early 1940s – prior to the first well publicized randomized controlled trial. Physicians can also understand and benefit from observations made centuries ago e.g. in physical diagnosis. So, if we adopt incommensurability as the indicator of a revolutionary paradigm shift, then we have no reason to say that EBM constitutes such a shift. The second problem with Kuhn's talk of incommensurability is that it is by no means obvious that there have been any new scientific theories that can be seen as genuinely incommensurable with the previous theories. After all, when Kuhn himself describes various alleged paradigm shifts, he always manages to tell us about both the old and the new scientific theories. Were his theory correct, this means he would have succeeded in doing that which is impossible. While the issues are too deep and complicated to be adequately addressed here, we will just say that Kuhn and his followers have not convinced us that the notion of incommensurable paradigms is even coherent.[22] Why it matters It seems to us that when proponents grandly describe EBM as a paradigm shift, they not only make a philosophical mistake, but they inadvertently have an unfortunate effect on the debate about the first order questions, for talk of paradigm shifts unduly polarizes the debate about the value of EBM.[23] Recall that Kuhn describes a paradigm as the "entire constellation of beliefs, values, techniques, and so on shared by the members of a given community".[19] When proponents of EBM suggest that it is a new paradigm, this certainly fosters the impression that an entire set of beliefs, values, and techniques is to be left behind, and that the whole world of medical research and clinical practice is completely different than it was in the days before EBM was recognized. This in turn strongly suggests that physicians have a stark choice: accept the new regime and completely reject the old, or defensively hold on to the old and reject EBM entirely. If something as grand as a complete shift of paradigm is in the offing, then the old physicians, with their old understandings,Page 4 of 10 (page number not for citation purposes) BMC Health Services Research 2003, 3 http://www.biomedcentral.com/1472-6963/3/14training, and paradigm, are relics to be removed as soon as possible. The prospect of this kind of consequence would certainly lead to a defensive retrenchment and even anxiety. If you exaggerate the claims for EBM, you intensify the natural defensiveness of those who feel themselves not to be completely in line with the new regime. This is not a productive atmosphere in which to hold a debate about the merits of EBM. In the next section, we suggest a different philosophical framework with which to examine the EBM debate. But we think that some progress would be made simply by avoiding overly broad definitions of EBM and by abandoning all talk of paradigm shifts. Quine and the web of belief We can make more progress by turning away from Kuhn and turning to the work of philosopher of science, W.V. Quine. Quine describes the totality of our beliefs – from ordinary common sense claims to the laws of physics – as a web, and he thus speaks of the web of belief. [12] A web, e.g., a spider web, has an exterior edge or frame, and an interior, consisting of radii and connecting points. At the edges, the web is secured to an already existing structure, e.g., a branch or wall. According to the analogy, the web of belief comprises statements; certain of the statements are closer to the periphery or frame, which consists of our sensory experience. Those sentences that lie closest to the periphery of the web Quine calls observation sentences, while the interior of the web contains more theoretical claims. The observation sentences are those that are more likely to be denied or affirmed on the basis of immediate sensory experience. For example, a quick glance out of the window will determine whether or not we accept the sentence "It is raining." With other sentences, we are much less immediately influenced by sensory experiences. For example, consider the sentence, "Ronald Reagan was President in 1987." One might have a sensory experience that is apparently at odds with this sentence; one might see a list that includes Walter Mondale as the 41st President serving from 1985 through 1988. But we would not allow such apparently recalcitrant sensory experience to change our minds about the original sentence. We would conclude instead that the list was simply wrong. This is not to say that the sentence "Ronald Reagan was president in 1987" is immune from all possibility of doubt; sufficient evidence might indeed convince us that the sentence is false. But two points are in order. First, it would take much more than a simple one-time observation of anything, and, second, we would have to change many other beliefs as well. This is characteristic of theoretical sentences within the sciences too: to convince us of the falsity of a firmly accepted claim, we will need repeated experiments with congruent results, and rejection of theoretical sentences typically requires the revision of many other related beliefs. Some theoretical claims will be even closer to the center of the web, e.g., laws of nature like "E=mc2" and laws of logic or mathematics. According to Quine, even claims as central as these are revisable. But because such claims are connected to so many other beliefs, their rejection would typically require revision to many, many sentences within the web. So according to the Quinean metaphor, we use a vast network or web of beliefs with intricate logical and evidential relations; at the periphery of the web are the observation sentences, which represent claims that are the most dependent on sensory observation. In the medical context, observation sentences would include simple findings during a physical examination, e.g., a patient's temperature or blood pressure. Moving inward from the periphery of the web would be hypotheses about these findings, e.g., that the patient has an infection, or that if the systolic pressure is 160 mmHg, we should repeat the measurement before concluding that it represents that patient's normal blood pressure. Even further towards the interior of the web would be theories about disease occurrence, e.g., that HIV is a retrovirus that targets CD4(+) T cells; that the differential diagnosis of hypertension includes idiopathic, renal stenosis and even pheochromocytoma; even further in the interior would be very general claims, e.g., the physiological model for elevated blood pressure or the germ theory of disease. It is important to note that Quine is not suggesting a foundationalist picture, according to which we build our theory on supposedly indubitable reports of sensory experience. Quine writes: Any statement can be held true come what may, if we make drastic enough adjustments elsewhere in the system. Even a statement very close to the periphery can be held true in the face of recalcitrant experience by pleading hallucination or by amending certain statements of the kind called logical laws.[12] One of Quine's central claims is "that our statements about the external world face the tribunal of sense experience not individually but only as a corporate body".[12] Thus our theories are put to the test as a whole; we cannot simply test individual claims in isolation from the rest of our theory. This Quinean view, which seems to us perfectly correct, is sometimes referred to as holism, or as the Quine-Duhem thesis (for Pierre Duhem had argued for a similar view.[24]). Nonetheless, observation sentences do have a certain privileged status. Observation sentences are those that are most directly tied to our immediate sensory experience;Page 5 of 10 (page number not for citation purposes) BMC Health Services Research 2003, 3 http://www.biomedcentral.com/1472-6963/3/14given the right sensory experience, all competent observers fluent in our language will unhesitatingly assent to the appropriate observation sentence. For example, while looking at a thermometer, all competent observers could agree that it reads 37.0 C.; and given the right visual, aural, and tactile stimuli while using a sphygmomanometer, all competent medical professionals could agree that the patient's blood pressure is 130/80 mmHg at the time of the measurement. Having made such observations, we will also be very reluctant to change our minds about the truth of those sentences. Of course, we might well come to conclude that the patient's blood pressure has changed, or that the reading does not represent the patient's normal blood pressure (if, for example, we had reason to think that the patient was under stress at the time). However, having made our observation, we will remain quite confident that the original reading was correct at the time. Naturally, we might be forced by other observations to conclude that we had misread the sphygmomanometer, the cuff was wrongly inflated, or even that the instrument was not functioning properly. But in general, we will be very reluctant to reject a previously accepted observation sentence. Our developing scientific theory must somehow accommodate the observation sentences we accept. In this sense, sensory experience and observation sentences are the ultimate checkpoint for scientific theory; they are that which secures the web to fixed ground. Application to EBM and the alternatives EBM is principally concerned with what we have called intervention questions of the form, 'is X safe and effective for condition Y?'. Answers to these questions will fit somewhere in the web of belief. They will clearly not be located at the very center of the web like principles of mathematics or laws of physics; in the face of recalcitrant empirical data (e.g., data from RCTs) we will revise our views about treatment efficacy much more readily than we will reconsider what we take to be basic laws of nature. On the other hand, answers to intervention questions will also not be at the very periphery of the web, for questions about the safety or efficacy of therapy are also clearly not answerable by simple observation statements. With something as complex as the human body, we will not be able to simply observe the exact effect of a particular therapy. (Of course, some people fail to understand this. Some people think that they can just see that, e.g., their breast implant caused some awful condition that developed subsequently. [25]) So questions about treatment efficacy and safety will not be on the outer periphery of the web, though they will also not lie too deeply in the interior. What is the relationship between EBM and its alternatives, as seen from the perspective of the Quinean metaphor of the web of belief? The answer depends on which of the alternatives we are considering. We'll first look at the third alternative to EBM, namely, deduction from basic scientific knowledge. EBM and Basic Science From our knowledge of human physiology, disease, and pharmacology we might be able to infer whether a particular drug would be effective in treating a given condition. With the basic science approach, we work up from our knowledge of physiology and biochemistry to a prediction of what will happen. Clinical trials essentially ignore, or suspend temporarily, all of that and determine whether a treatment works by trying the treatment in a large number of cases under controlled conditions. Basic science and RCTs thus represent two different paths to answering intervention questions. Seen explicitly in terms of the web metaphor, RCTs seem to take the shortest path possible from the periphery of the web towards the mid-web space where clinical decisions are made. Within the course of a clinical trial, we will make observation statements that seem very directly relevant to our original treatment question, for we will be observing clinical endpoints for other patients with condition Y when given treatment X. The basic science approach, on the other hand, looks rather different. On the basis of biochemistry and physiology, we might predict certain clinical endpoints for patients with condition Y when given treatment X. These predictions will be based on a very different set of observation statements than those involved in RCTs. The observation statements that ultimately support the basic science approach will be those that support our knowledge of biochemistry and physiology. Although EBM and the basic sciences embody different approaches, this does not mean that they are competitors. In fact, the two approaches need each other; neither can stand alone. This is perhaps easiest to see with basic science. If basic science could give us perfect confidence in the safety and efficacy of an intervention, then clinical trials would indeed be theoretically superfluous. However, we rarely, if ever, can be certain of both the safety and efficacy of a treatment without clinical testing, for our knowledge of the human body and how it interacts with the environment is far from complete. While it is perhaps less obvious, the method of clinical trials is also not able to stand alone. RCTs can sometimes give us confident answers to intervention questions even when our basic scientific knowledge is insufficient for this purpose (e.g., on many questions concerning diet and exercise); however, this does not mean that RCTs are completely independent of basic science. If nothing else, a controlled trial needs a hypothesis to test. Typically, treatments will be suggested as candidates because of other more basic scientific research. A classic case of this is thePage 6 of 10 (page number not for citation purposes) BMC Health Services Research 2003, 3 http://www.biomedcentral.com/1472-6963/3/14accidental discovery of penicillin by Alexander Fleming, though Fleming himself initially paid little attention to the possible therapeutic uses of the mold.[26] Contemporary basic scientific work on thalidomide provides another example.[27] But there is also a more fundamental way in which RCTs cannot stand apart from basic science. Even when a clinical trial returns positive results in the treatment arm that satisfy tests of statistical significance, we will have more confidence in these results when they have some antecedent biological plausibility.[28,29] Put more generally, we would suggest that the degree of confidence appropriate for a clinically tested claim is a function of both the strength of the clinical result and the claim's antecedent biological plausibility. This relationship is perhaps obscured by cases in which we understand very little about the pharmacology of a drug (e.g., treatment of ulcerative colitis with sulfasalazine). In such cases, it is perfectly appropriate that strong clinical results yield a reasonably high degree of confidence. But we shouldn't forget that we would be even more confident if the results were exactly what biochemistry and physiology would lead us to expect. More importantly yet, if the basic sciences gave us very strong reason to believe that a drug would not be effective, then it is appropriate to be very cautious when interpreting apparently positive clinical results. For example, there have been some RCTs of homeopathic remedies according to which subjects receiving the homeopathic remedy did better than those receiving a placebo, and where the P-values are less than .05 or even less than .001.[30] However, given that homeopathic remedies are typically composed of nothing more than water, accepted basic science should lead us to seriously doubt that these 'drugs' can be effective. In any event, we cannot simply accept that pure water (in the form of a homeopathic medicine) is medically effective unless we are also willing to revise much of our basic scientific conception of chemistry and the body. The interior of the web acts as a compass we use to interpret the direction we should take from signals from a new RCT. Statistical information from an RCT is virtually uninterpretable and meaningless if stripped away from the backdrop of our basic understanding of physiology and biochemistry. The dependence of RCTs upon the backdrop of basic science is unsurprising from the Quinean perspective. The point is essentially an application of the Quinean doctrine of holism, the claim that our theories are put to the test as whole bodies rather than being tested sentence by sentence. (Tonelli and Callahan [31] make a similar point about the need for theory in interpreting empirical results. However, they suggest that evidence concerning alternative medicine should be weighed and interpreted in terms of its theory of disease, e.g., theories involving undetectable spirit essences or the like. This does not follow at all. If we are judging the effectiveness of alternative medicinal therapies, then we evaluate the evidence in light of the theory about disease that we believe to be true. It would be a silly and pointless exercise to evaluate evidence in light of a theory we believe to be false.) The dependence of RCTs on basic science appears not always to have been fully appreciated by some EBM proponents. In some discussions of hierarchies of medical evidence, basic physiology and biochemistry is put at the very bottom or is ignored altogether.[14,32] Thus, these proponents of EBM seem to say that we should only look at evidence from RCTs (when we have it), and that medical practice is conducted in isolation of our understanding of physiology and biochemistry. We claim that such views are wrong. On the other hand, this is not to say that if a clinical result flies in the face of our physiological understanding that we should just reject it out of hand, as suggested by Couto [5]. In general, given the complexity of the human body and our nascent knowledge of it, if we use basic science alone to judge the safety or efficacy of an intervention, then our judgment will be fraught with a high degree of uncertainty. For example, some laboratory evidence suggests that estrogen might have a positive effect on brain cells, and thus suggesting that hormone therapy might help to ward off dementia. However, a recent controlled trial strongly suggests that whatever positive effect such hormones have, these effects are outweighed by negative effects.[33] On the other hand, in an extreme case like homeopathy, where the biological plausibility is so low, our background biological knowledge does and should affect our interpretation of the evidence from RCTs. The outcome of our discussion above is that basic science and EBM are deeply intertwined within the web and complementary. In Quinean terms, each of these represents a different path to the intervention question through the web of belief. We clearly need both approaches. Which of the approaches should receive the greater share of our limited health care resources? There is little useful to be said about this question at this level of generality. The best allocation of resources is affected by a broad range of contextual factors. EBM and the other alternatives As noted earlier in the discussion of Kuhn, clinical experience, observational studies, and RCTs have much in common. All are attempting to ascertain the safety and efficacy of interventions, and all do so by trying the intervention and noting the results. From the Quinean perspective, we could also say that all three are anchored to observation statements at the periphery of the web in similar ways. In each case, we will observe the treatment received by aPage 7 of 10 (page number not for citation purposes) BMC Health Services Research 2003, 3 http://www.biomedcentral.com/1472-6963/3/14patient, and then we will observe the outcome or endpoint for each patient. Of course, in an RCT, we place restrictions on the way data are gathered: we require that there be a control group of patients who do not receive the target experimental therapy, and that patients are randomly assigned to either the control or experimental group; we typically 'blind' researchers (and other participants), by not allowing them to know, during the course of the study, which patients are in the control group; and we blind the patients themselves, so that they do not know which therapy they received. In this section, we will look at how these restrictions are to be understood and justified from the perspective of the Quinean metaphor. In observational studies or clinical experience, the physician knows which subjects received the therapy or exposure, and then observes the patient's subsequent condition. Thus, statements about the patient's condition constitute the evidential base, and these statements are treated as observation sentences. But recall that an observation sentence is one that would elicit assent from any competent observer. The advocate of double-blind trials contends, in effect, that simple statements about the patients' condition are not observation sentences after all, for the sentence might not elicit assent from all competent observers – especially for 'soft' end points such as pain scales. Physicians may be unconsciously biased in favor of seeing or eliciting a treatment effect when none exists. A doctor who knows that the patient has received a certain therapy, and who thinks that the therapy is likely to be effective, might make observations that would not be made by a physician who knew nothing of the treatment. So the criticism of clinical experience and observational studies is this: the evidence in those cases is tainted in that physicians take certain sentences about the patient's condition to be observation sentences (and hence firm checkpoints) when in fact they do not qualify as observation sentences. Hence, according to the proponent of EBM, the conclusions reached through clinical experience are less reliable. In a double-blind trial, we also blind patients from information about what sort of therapy they received. Here, the main concern is not that the patients will be subconsciously biased in reporting their condition – though this might be a concern in some cases. The issue is very different: we want to rule out placebo effects (or the "meaning response" [34]), meaning that we want exclude the possibility that the observed effect stemmed from the patient's belief that she was given a particular therapy. We might have reliable observation sentences about the patient's subsequent condition, but, unless we control for placebo effects, the improvement in a patient's condition might have nothing to do with the actual treatment received. In Quinean terms, in an uncontrolled study, the observation statements (about the patient's condition) and the theoretical statement (about the efficacy of the treatment) may be linked too tenuously along the radii of the web; in the extreme case, the apparent linkage may be purely by chance. If we control the conditions under which we make observations, we can then compare the observation statements concerning the condition of patients who received the treatment with statements concerning patients who received a placebo. In other words, we use observation sentences similar to those that would be obtained in clinical experience, but we are more careful in the inferences we draw from these statements. The EBM proponent contends that by including observations about patients in the control group, we put ourselves on firmer ground in determining the efficacy of the treatment; thus the web is strengthened. Perhaps the most basic feature of an RCT is the use of randomized control groups. When research is done without using control groups at all, researchers might infer treatment was effective, when in fact the patient's condition would have waned on its own or would have improved just as much with the standard therapy. In other words, the EBM proponent claims that in the absence of a control group, the inference from observation statements (about the patient's condition) to the conclusion (about the efficacy of the intervention) is greatly weakened. Moreover, if we do not randomly assign patients to either the experimental or control groups, then we again run the risk of researcher bias affecting the outcome, e.g., by selecting patients for the experimental group who have the best chance of improvement. Returning to the first-order, normative question, we can now ask to what extent should physicians rely on RCTs as opposed to clinical experience or observational studies. Unlike the comparison with basic science, we cannot simply say that they are complementary and that we need both. As discussed above, observational studies essentially are clinical experience made systematic, and RCTs are observational studies subject to further controls. The extra time and expense involved in performing RCTs is justified only if their answers to intervention questions are more likely to be true. For the reasons laid out in the preceding paragraphs, the proponent of EBM contends that RCTs are indeed more reliable than the alternatives. We are inclined to agree. It is clear that for clinical experience and observational studies, there is a risk of different sorts of bias; in Quinean terms, the controls imposed by RCTs give us firmer observational checkpoints and stronger inferences from observations to conclusions, particular conclusions about interventions. Experience confirms that uncontrolled studies can give different results than RCTs [35,36]; the Quinean analysis gives a diagnosis of those differences.Page 8 of 10 (page number not for citation purposes) BMC Health Services Research 2003, 3 http://www.biomedcentral.com/1472-6963/3/14Still, that doesn't mean we should reject clinical experience or observational studies altogether. Much of our web of medical theory was spun from these sources. Moreover, as a number of authors have pointed out, there are significant limitations on when RCTs are practical or even ethical. Where limitations of this sort intervene, we have to do the best we can, and that means relying on the experience we have through observational studies and clinical experience, even though we know that there may be problems of bias and selective memory, etc. If there are EBM proponents who thinks that medical practice can be reduced to an algorithmic application of evidence from RCTs, they are certainly mistaken.[37] Summary Because we wished to clarify the debate about EBM, we have spent the bulk of this paper exploring the secondorder conceptual questions concerning the nature of EBM. We have argued that it is both a philosophical and practical mistake to see EBM as a paradigm shift. We have also attempted to explain the differences between EBM and its alternatives in terms of Quine's metaphor of the web of belief. EBM and the basic sciences offer different paths through the web to claims about intervention safety and efficacy: each begins with a very different set of observation statements as its evidential checkpoint; however, neither is completely self-standing and independent. There is a sharper contrast between EBM and the methods of clinical experience and observational studies, according to the EBM proponent: uncontrolled studies take certain reports of clinical endpoints as observation sentences – i.e., sentences that would be agreed to by all competent observers, sentences that are the checkpoint and basis for science – when in fact some of those reports do not count as observation sentences; and in other cases, uncontrolled studies may base their conclusions on observation sentences that are too tenuously linked to the claimed result, resulting in a post hoc ergo propter hoc fallacy. This does not mean that RCTs are always appropriate or are always feasible, and there is much more to be said about the first order questions of when RCTs are worth the time and expense. But we can only have that discussion when we are clear about the nature of the questions themselves. Competing Interests None declared. Acknowledgments We would like to thank Dr. Henrik Wulff and Dr. Ross Upshur for detailed comments on earlier drafts of this paper. References 1. Guyatt GH, Haynes B, Jaeschke R, Cook D, Greenhalgh T, Meade M, Green L, Naylor CD, Wilson M, McAlister F and Richardson WS: Introduction: The Philosophy of Evidence-Based Medicine Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice Edited by: Guyatt G and Rennie D. Chicago, AMA Press; 2002:3-12. 2. Eisenberg John M.: Evidence-Based Medicine Expert Voices 2001. 3. Muney Alan M.: Evidence-Based Medicine Needs To Be Promoted More Vigorously Managed Care 2002. 4. Goodman NW: Who will challenge evidence-based medicine? J R Coll Physicians Lond 1999, 33:249-251. 5. Couto JS: Evidence-based medicine: a Kuhnian perspective of a transvestite non-theory Journal of Evaluation in Clinical Practice 1998, 4:267-275. 6. Shaughnessy AF, Slawson DC and Becker L: Clinical jazz: harmonizing clinical experience and evidence-based medicine J Fam Pract 1998, 47:425-428. 7. Charlton BG and Miles A: The rise and fall of EBM QJM 1998, 91:371-374. 8. Tanenbaum SJ: What physicians know The New England Journal of Medicine 1993, 329:1268-1271. 9. Miles A., Grey J.: New perspectives in the evidence-based healthcare debate Journal of Evaluation in Clinical Practice 2000, 6:7784. 10. Miettinen OS: The modern scientific physician: 1. Can practice be science Canadian Medial Association Journal 2001, 165:441442. 11. Benson K and Hartz AJ: A comparison of observational studies and randomized, controlled trials Am J Ophthalmol 2000, 130:688. 12. Quine WV: From a Logical Point of View 2ndth edition. Cambridge, MA, Harvard University Press; 1953. 13. Tonelli MR: The philosophical limits of evidence-based medicine Academic Medicine 1998, 73:1234-1240. 14. Sackett DL, Straus SE, Richardson WS, Rosenberg W and Haynes RB: Evidence-Based Medicine: How to Practice and Teach EBM 2ndth edition. Edinburgh, Churchill Livingstone; 2000. 15. Haynes RB: What kind of evidence is it that Evidence-Based Medicine advocates want health care providers and consumers to pay attention to? BMC Health Serv Res 2002, 2:3. 16. Haynes RB, Devereaux PJ and Guyatt GH: Physicians' and patients' choiced in evidence based practice British Medical Journal 2002, 324:1350. 17. Haynes B, Devereaux PJ and Guyatt G: Clinical expertise in the era of evidence-based medicine and patient choice APC Journal Club 2002, 136:A11-A14. 18. Guyatt GH, Meade MO, Jaeschke R, Cook D and Haynes B: Practitioners of evidence based care British Medical Journal 2000, 320:954-955. 19. Kuhn Thomas S.: The structure of scientific revolutions 2d, enl.th edition. Chicago, University of Chicago press.; 1970:xii, 210. 20. Guyatt GH and Rennie D: Users' Guide to the Medical Literature: A Manual for Evidence-Based Clinical Practice Chicago, AMA Press; 2002. 21. Wulff HR and Gotzsche PC: Rational Diagnosis and Treatment: Evidence-Based Clinical Decision Making 3rdth edition. Blackwell Publishers; 2000. 22. Davidson D: On the Very Idea of a Conceptual Scheme Proceedings and Addresses of the American Philosophical Association 1974, 47:520. 23. Miles A., Bentey, P., Polychronis A., Grey J.: Evidence-based medicine: why all the fuss? This is why. Journal of Evaluation in Clinical Practice 1997, 3:83-86. 24. Duhem Pierre: La Theorie physique: son objet et sa structure Paris; 1906. 25. Angell Marcia: Science on Trial: The Clash of Medical Evidence and the Law in the Breast Implant Case W.W. Norton & Company, Inc.; 1996. 26. Kiester E: A Curiousity Turned into the First Silver Bullet Against Death Smithsonian 1990, 21:173-188. 27. D'Amato RJ, Loughnan MS, Flynn E and Folkman J: Thalidomide is an inhibitor of angiogenesis Proc Natl Acad Sci U S A 1994, 91:4082-4085. 28. Goodman Steven N.: Towards Evidence-Based Medical Statistics. 1: The P Value Fallacy Annals of Internal Medicine 1999, 130:995-1004. 29. Goodman Steven N.: Toward Evidence-Based Medical Statistics. 2: The Bayes Factor Annals of Internal Medicine 1999, 130:1005-1013.Page 9 of 10 (page number not for citation purposes) BMC Health Services Research 2003, 3 http://www.biomedcentral.com/1472-6963/3/14Publish with BioMed Central and every scientist can read your work free of charge "BioMed Central will be the most significant development for disseminating the results of biomedical research in our lifetime." Sir Paul Nurse, Cancer Research UK Your research papers will be: available free of charge to the entire biomedical community peer reviewed and published immediately upon acceptance cited in PubMed and archived on PubMed Central yours - you keep the copyright Submit your manuscript here: http://www.biomedcentral.com/info/publishing_adv.asp BioMedcentral 30. Linde K, Clausius N, Ramirez G, Melchart D, Eitel F, Hedges LV and Jonas WB: Are the clinical effects of homeopathy placebo effects? A meta-analysis of placebo-controlled trials Lancet 1997, 350:834-843. 31. Tonelli M.R., Callahan, T.C.: Why alternative medicine cannot be evidence-based Academic Medicine 2001, 76:1213-1220. 32. Centre for Evidence-Based Medicine: Levels of evidence and grades of recommendation 2003, 2003: [http://www.cebm.net/ levels_of_evidence.asp]. Centre for Evidence-Based Medicine 33. Shumaker SA Legault C, Thal L, Wallace RB, Ockene JK, Hendrix SL, Jones BN 3rd, Assaf AR, Jackson RD, Morley Kotchen J, WassertheilSmoller S, Wactawski-Wende J: Estrogen Plus Progestin and the Incidence of Dementia and Mild Cognitive Impairment in Postmenopausal Women: The Women's Health Initiative Memory Study: A Randomized Controlled Trial. JAMA 2003, 289:2651-2662. 34. Moerman Daniel E. and Jonas Wayne B.: Deconstructing the Placebo Effect and Finding the Meaning Response Annals of Internal Medicine 2002, 136:471-476. 35. Schulz K.F., Chalmers, I., Hayes, R.J., Altman, D.G.: Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995, 273:408-412. 36. Guyatt G: Therapy and harm: Why study results mislead-bias and random error Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice Edited by: Guyatt G Rennie D. Chicago, AMA Press; 2002:223-229. 37. Upshur RE: Seven characteristics of medical evidence Journal of Evaluation in Clinical Practice 2000, 6:93-97. Pre-publication history The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1472-6963/3/14/prepubPage 10 of 10 (page number not for citation purposes)


Philosophy

Chapter 19 Algorithms and Arguments: The Foundational Role of the ATAI-question Paola Cantù and Italo Testa Centre d'Epistémologie et Ergologie Comparative, Université de Provence, France Dipartimento di Filosofia, Università di Parma, Italy 1. Introduction Argumentation theory underwent a significant development in the Fifties and Sixties: its revival is usually connected to Perelman's criticism of formal logic and the development of informal logic. Interestingly enough it was during this period that Artificial Intelligence was developed, which defended the following thesis (from now on referred to as the AI-thesis): human reasoning can be emulated by machines. The paper suggests a reconstruction of the opposition between formal and informal logic as a move against a premise of an argument for the AI-thesis, and suggests making a distinction between a broad and a narrow notion of algorithm that might be used to reformulate the question as a foundational problem for argumentation theory. The paper starts by the analysis of an argument in favor of the AI-thesis (from now on referred to as the AI-argument), distinguishing three premises that support the conclusion (§ 2). We suggest that the interpretation of informal logic as strictly opposed to formal logic might be interestingly analyzed as a move in a strategy to refute the AIthesis by attacking a premise of the argument: the possibility of expressing arguments by means of algorithms. We are not thereby suggesting that this move was explicitly made by argumentation theorists; nonetheless this counterfactual reconstruction might shed some light on the reasons that opposed argumentation theorists and AI-scholars. In particular, we suggest that the opposition between a formal and an informal approach need not be interpreted only as a way to deal with the peculiarities of ordinary language (analytic philosophy of language answered a similar need without renouncing formal tools, even if only fragments of the natural languages could be formalized), but might also be considered as a way to distinguish the domain of human argumentative rationality from the domain of mechanical computation. The mentioned strategy will then be compared with other moves directed at the rebuttal of the conclusion of the argument (§ 3). This will allow to distinguish the criticism of the possibility of expressing arguments by means of algorithms from the criticism of the interpretation of Leibniz's logical calculus as the structure of human reasoning, and from the criticism of the thesis that all computable functions can be calculated by a Turing-machine. The comparison of different strategies to rebut the conclusion of the argument will show that a certain understanding of the notion of algorithm is essential in all three strategies: algorithms are considered as computable functions. We will afterwards discuss a broader notion of algorithm that is often referred to in the literature either as a more intuitive and primitive notion or as a notion that needs to be developed in order to ground recent developments in computation theory and AI (§ 4). We will interpret the narrow notion of algorithm (algorithms are computable 193 functions) as a formal definition that applies only in certain cases but that can fruitfully contribute to an understanding of the intuitive notion. We will suggest a general characterization of the broad notion as an enlargement of the narrow notion of algorithm. The latter is based on the definitions given by Markov and Knuth (§ 5). Common features of the two notions are finiteness, generality, conclusiveness, while some relevant differences concern the formulation of effectiveness, which needs to be loosened, definiteness, and determinism, which need to be abandoned if one wants to include non-deterministic algorithms, or indefinite algorithms that need to be interpreted by the receiver in a given context, or more generally algorithms that cannot be computed by a Turing-machine. We will then consider a distinction between a broad and a narrow notion of argument (§ 6), suggesting that, if one interprets formal logic as a sub-domain of informal logic rather than as a radically incompatible research area, then the broad notion of argument can be considered as more primitive and the narrow notion can be seen as a restriction that is useful to understand the nature of arguments but that is also insufficient for certain purposes of argument analysis. Given this interpretation of the relations between formal and informal logic, several similarities between the broad notions of argument and algorithm are considered (§ 7): not only the history of the relations between a broad and a narrow notion is similar in the two cases, but the two broad notions can be similarly described by difference with respect to the two narrow notions: the former are informal rather than formal, pragmatic rather than only syntactic, in need of an interpretation rather than unambiguously determined, non-deterministic rather than deterministic. The distinction between a broad and a narrow notion of algorithm will also explain why it was so easy for argumentation theorists to refute the idea that arguments could be expressed by algorithms: they were comparing the broad notion of argument with the narrow notion of algorithm. Once the comparison is made between the two broad notions, certain similarities cannot be ignored, and the fruitfulness of the application of AI to argumentation might be investigated anew. In the last section of the paper (§ 8) we will go back to the argument sketched out in § 2 in order to claim that the distinction between a broad and a narrow notion of argument, and the developments made by logic, computation theory, AI and argumentation theory in recent years make it easy to rebut the conclusion of the argument. But maybe that is only due to the fact that the idea expressed by it needs to be reformulated in the light of those developments: the question suggested by AI does not concern the emulation of the argumentative reasoning of a single human mind, but rather the emulation of the argumentative practices of several interlocutors interacting with each other in a given context. The question would now be whether a multi-agent system can emulate the interactive reasoning of several human participants in a discussion (from now on referred to as the ATAI-question). This paper does not aim to give a definite answer to the problem, but considers it as a leading idea in the application of AI to argumentation theory and as an open question that is not limited to logic or philosophy of mind but that involves the foundations of argumentation theory itself, and especially its conception of argumentative rationality. 2. The AI-argument and its criticism by argumentation theorists Between the end of the Fifties and the beginning of the Sixties research into formal logic and AI were oriented by the idea that 194 (1) human reasoning can be considered as a mechanical computation (Leibniz's calculemus). The majority of AI scholars also believed in the so-called Church-Turing thesis, which can be roughly formulated as follows: (2) any computable function can be computed by a Turing-machine.1 So, if one accepts the further premise that (3) arguments can be reconstructed as algorithms, then one can infer by means of (1), (2) and (3) that (AI-thesis) human argumentative reasoning can be emulated by a machine. It is well known that a main reason for the revival and development of argumentation theory in the Fifties and Sixties was the reaction to the neopositivist ideas that there could be no rational discussion on judgements of value and that logic could be conceived as a mathematical calculus rather than as a general theory of human reasoning. We would like to suggest that there was a third element of disagreement between argumentation theorists and formal logicians: it concerned the role attributed to algorithms in the representation and understanding of human reasoning.2 AI scholars believed that human reasoning was a mechanical computation (1) and thus aimed at restricting the notion of algorithm so as to identify it with a class of computable functions. According to our interpretation the insistence on the opposition between formal and informal arguments could be seen, in the light of recent developments of AI, and independently from the intentions of the argumentation theory scholars that first defended such an opposition, as a move against the AI-argument. Assuming the Church-Turing thesis (2) to be valid, and assuming that arguments can be reduced to algorithms (3), one could derive the conclusion that human reasoning can be emulated by a machine (AI-thesis). But if this is true, there would be no space left for the specific human "rationality" of argumentation. So, while attacking premise (3), one would at the same time rebut the AI-argument, if not attack the AI-thesis altogether. When arguments are defined as classes of sentences of the natural language that could not be adequately translated into any formal language, then they are defined by opposition to algorithms. Besides, it is not uncommon in the argumentation theory tradition to strongly criticize the reduction of arguments to deductive inferential schemes. So, even if we are not suggesting that any argumentation scholar has explicitly advocated this strategy, some of them might agree on the premises of the argument and might be satisfied with its conclusion. The strategy consisting in the denial of the AI-thesis by refuting premise (3) was useful to distinguish argumentation theory from logic, and thus a condition for the existence of argumentation theory itself, given that if human reasoning does not differ substantially from the reasoning of a machine, there would be no need to distinguish the domain of human rationality from the domain of formal logic (Govier 1987, pp. 204-5). 195 3. Other strategies to attack the conclusion of the AI-argument Whether all human reasoning could be emulated by a machine, and whether there was nothing in the human mind that could exceed the powers of a calculating machine became main philosophical questions in logic and philosophy of mind. Among those who tried to refute the AI-thesis there were not only argumentation theorists, but also philosophers and logicians. The move made by argumentation theorists was not the only possible one. Other possible moves included the attack on premise (2), i.e. on the Church-Turing thesis, or on premise (1), i.e. on the mechanical conception of logical reasoning. Kurt Gödel for example criticized the Church-Turing thesis in a remark on undecidability results, where he reacted to the following version of the thesis: Turing machines can compute any function "calculable by finite means" (Turing 1937, p. 250). There is a huge body of literature discussing the meaning of Gödel's remark although in this paper we will not go into details. What is relevant here is the generally accepted fact that Gödel intended to suggest counterarguments to the idea that the generalized undecidability results might establish bounds for the powers of human reason (Gödel 1986, p. 370). Furthermore, it is relevant that he considered Turing's argument "which is supposed to show that mental procedures cannot go beyond mechanical procedures", as not yet conclusive, because "what Turing disregards completely is the fact that mind, in its use, is not static, but constantly developing, i.e. that we understand abstract terms more and more precisely as we go on using them, and that more and more abstract terms enter the sphere of our understanding. [...] This process, however, today is far from being sufficiently understood to form a well-defined procedure." (Gödel 1972a, p. 306). Even if we admit premise (1), i.e. that human reasoning is a mechanical procedure, its calculations cannot yet be expressed by well-defined procedures. Another possible strategy to refute the AI-thesis consisted in the attack on premise (1). A similar move had been done already at the end of the 19th century by J. Venn, who argued that even if human reasoning were based on algorithms, it could not be considered as a mechanical computation: "There is, first, the statement of our data in accurate logical language. [...] Then secondly, we have to throw these statements into a form fit for the engine to work with–in this case the reduction of each proposition to its elementary denials. [...] Thirdly, there is the combination or further treatment of our premises after such reduction. Finally, the results have to be interpreted or read off. This last generally gives rise to much opening for skill and sagacity; [..] I cannot see that any machine can hope to help us except in the third of these steps; so that it seems very doubtful whether any thing of this sort really deserves the name of a logical engine" (Venn 1881, pp. 120-121). In his 1972 article on the extension of finitary mathematics Gödel interestingly remarked upon a difference between the definition of algorithm occurring in the formulation of Turing's thesis and the intuitive notion of a well-defined procedure or algorithm: the latter is a primitive notion. Although he considers it as adequately expressed by Turing's notion of a mechanically computable function, Gödel adds that "the phrase 'well-defined mathematical procedure' is to be accepted as having a clear meaning without any further explanation." (Gödel 1972, p. 275). It is interesting to remark that all three strategies are based on the common implicit premise that the conception of algorithms can be adequately described by the notion of computable functions. As Gödel somehow suggested, the notion of algorithm is nonetheless antecedent to Turing's definition and further developments of AI and computation theory have shown that the former might be broader than the latter. In the 196 next section (§ 4) we will thus consider a different understanding of the notion of algorithm that will require a new evaluation of similarities and differences between algorithms and arguments (§ 7). This will also imply that the attack of premise (3) in order to rebut the AI-thesis might not be easily made nowadays. 4. A broad and a narrow notion of algorithm Recent developments of computation theory and AI suggest that the intuitive notion of algorithm might be broader than the notion of a Turing-machine computable function. Firstly, there are some procedures that cannot be computed by a Turingmachine. Some of them can nonetheless be computed by other kinds of machines (Gurevich 2000, p. 77 ff.). If an algorithm could be defined as a function that can be computed by a broader class of machines, including the Turing-machine as a particular case, then this notion would be broader than the one given by Turing. Secondly, there are several notions of a computable function (lambdacomputable, general recursive, primitive recursive, partial functions, ....), and there is no definite evidence that the notion of algorithm should be adequately and uniquely expressed by one of them. As Gödel himself noted in the previously mentioned passages, an intuitive notion of algorithm precedes the notion of a computable function. Blass and Gurevich are even more radical: "it is often assumed that the Church-Turing thesis settled the problem of what an algorithm is. That isn't so. The thesis clarifies the notion of computable function. And there is more, much more to an algorithm than the function it computes. The thesis was a great step toward understanding algorithms, but it did not solve the problem what an algorithm is" (Blass and Gurevich 2003, p. 197). Thirdly, the definition of algorithm as a computable function was the result of efforts to formulate algorithms that can be computed in a reasonably short time and in a reliable way by machines, but the notion of algorithm historically preceded both the notion of function and the invention of calculating machines. As an example, one could mention the nine chapters on mathematical procedures by Liu Hui written at the beginning of the third century (Chemla 2005, p. 125). Similarly, in the common understanding of algorithms as recipes or procedures to carry out some task (Sipser 2006, p. 142), algorithms are sets of instructions written for human receivers. Unlike Turing-machines, the instructions given to a human receiver need not be completely unambiguous. The context of the algorithm and other pragmatic elements might help the receiver to interpret the instructions of the procedure. So conceived, algorithms might contain procedures that cannot be computed by a Turing-machine. Finally, the development of multi-agent systems in AI has favoured the investigation of interactive algorithms, that can be implemented on a network of machines: multi-agent systems that can learn from experience and interact in a network. The class of interactive algorithms is so broad as to include randomized algorithms, asynchronous algorithms, and non-deterministic algorithms as well. In other words it includes algorithms that "are not covered by Turing's analysis" (Blass and Gurevich 2003, p. 203). The analysis of the developments of mathematics, computation theory, and AI shows that a broader notion of algorithm not only preceded the formalized definition given in the 20th century, but has also been the object of research in computation theory. The need for a more precise notion of algorithm induced a narrowing of the notion in order to define it as a Turing-machine computable function. Later on some 197 computation theorists and AI researchers discovered that this definition might be too narrow to be applied to some interesting examples, and started to progressively broaden the notion of algorithm. We suggest that the narrow notion of algorithm might be conceived as a temporary restriction of a more intuitive and broader notion–a restriction that was particularly useful to understand and formalize certain aspects of the broader notion, but that does not pretend to include all kinds of algorithms. Rather than broadening the notion of an algorithm by enlarging the class of computable functions or the class of machines to which algorithms correspond – a strategy that has been followed for example by Gurevich – we want to develop here a conceptual analysis of the conditions that the narrow notion usually satisfies and that the broader notion might fail to satisfy. We will claim that a provisionary understanding of the broader notion of algorithm that is at stake in AI and in computation theory could be obtained from the narrow notion if one abandons the conditions of definiteness and determinism, and if one does not formulate effectiveness in a very strict way. If the broader notion of algorithm can be obtained by a modification of the definition of the narrow notion, this does not mean, as we have already suggested in the previous paragraphs, that the narrower notion should be more primitive: on the contrary, the broader notion precedes both historically and conceptually the narrower notion. The latter, though, is easier to formalize, and can thus be used as a starting point for the analysis of the former. 5. A conceptual analysis of the differences Our suggestion for a characterization of the narrow notion of algorithm is derived, with some modifications and integrations, from the definitions given by Markov and Knuth between the Fifties and the Sixties (Markov 1961 and Knuth 1997). An algorithm is a set of instructions determining a procedure that satisfies the six following conditions: finiteness, generality, conclusiveness, effectiveness, definiteness, and determinism. Finiteness expresses the fact that the procedure allows, given certain inputs, to reach the goal (decision, computation, problem solving), i.e. provide the desired output in a finite number of steps. Generality guarantees the possibility of starting out with initial data, which may vary within given limits (e.g. certain general classes of inputs are admitted). Conclusiveness expresses the fact that the algorithm is oriented towards some desired result which is indeed obtained in the end if proper initial data are given. Effectiveness requires that the operations to be performed are sufficiently basic that they can in principle be done exactly and in a finite length of time by the executer (e.g. a man using a paper and a pencil). Definiteness requires that the prescription should be universally comprehensible and precise, leaving no place for arbitrariness. Determinism guarantees that, given a particular input, the procedure will always produce the same output, and will consist in the same sequence of steps.3 The mentioned characterization determines a class of definitions of algorithms rather than being itself a definition of algorithm: differences might derive from specific or detailed formulations of each condition. Effectiveness might be for example intended as strongly or weakly polynomial-time complexity; generality might be specified as the requirement that all inputs belong to the class of natural numbers or to the class of real numbers, and so on. In the light of the brief survey of some occurrences of a broader notion of algorithm given in § 4, we suggest that the broad notion should maintain some features of the narrow notion, allowing other features to be formulated in a more liberal way or 198 abandoned altogether. In particular, the narrow notion of algorithm should be better characterized by finiteness, generality, conclusiveness, and by a 'liberal' formulation of effectiveness. This condition has nonetheless to be at least partially maintained if one wants the algorithm to be concretely computable by some kind of physical machine. The conditions of definiteness and determinism might be abandoned, so as to include non-deterministic algorithms, indefinite algorithms that need some interpretation by the receiver, and algorithms that cannot be computed by a Turing-machine. Abandoning these conditions need not mean of course that all parts of an algorithm would be nondefinite or non-deterministic: in order to preserve some kind of effectiveness, considerable portions of the algorithm might have to be definite and deterministic. 6. A narrow and a broad notion of argument After having introduced a distinction between a narrow and a broad notion of algorithm, we would now like to go back to the definition of argument. This will help a further understanding of premise (3), because in order to discuss if arguments can be expressed as algorithms one should consider which notion of algorithm and which notion of argument is at stake. In the history of argumentation theory several definitions of an argument have been given. A detailed list of different definitions cannot be presented here, but two main classes of definitions can be distinguished. The first class contains the definitions of what we will call the narrow notion of an argument, including the Aristotelian scientific syllogisms and formal representations of deductive inferences such as Lorenzen's dialogical moves. Common characteristics of this narrow notion of argument are the formal representation, the central role played by deduction as a core inference, and the context-independent definition of validity. The second class contains several definitions that express a broader notion of argument, including for example the pragmatic conception developed by van Eemeren and Grootendorst (2004), and the informal notion of argumentative schemes developed by Perelman. The definitions that belong to this class are usually informal, context-dependent and based on a diversification of the kinds of relations that can occur between premises and conclusions in order for the argument to be valid: deductive and inductive inferences, but also other schemes, such as analogy or causal relation, are admitted as valid. The relation between the two classes of definitions can be conceived differently (Johnson & Blair 2002, p. 357 and D'Agostini 2010, p. 35). Some authors consider them as two complementary classes: the informal definition of argument is opposed to the formal notion, as if the two concepts were radically different and applied to different domains (Scriven 1980). Other authors conceive the broad notion as an enlargement of the narrow notion that might be partially or wholly formalized by means of more sophisticated logical tools (non-monotonic logic, dialogue logic, default logic, defeasibility, and so on) (Woods et al. 2002). Following this second interpretation of the relations between the two notions, we have elsewhere argued (Cantù & Testa 2006, pp. 18-21) that the narrow notion might be considered as a temporary restriction of the broader notion that is useful to better understand the notion of inference, rather than as a concept that is radically opposed to it. In our reading, the opposites informal/formal, syntax/pragmatics, and deductive/non-deductive can be read as relations of subordination rather than as relations of contrariety, and informal logic is considered as an enlargement or liberalization of formal logic. Arguments expressed in the natural language are thus 199 informal not in the sense that they cannot be formal, but rather in the sense that they are "only partially formalizable" by means of the logical tools at our disposal. The narrow notion is in fact useful to formalize certain arguments that fall under the broader notion, or at least certain parts of them (Woods & Walton 1982), as well as the formal notion of argument can be used to better understand an argumentation that can never be fully articulated in the natural language, or at least not in the same way. 7. Similarities between arguments and algorithms Given this interpretation of the relations between formal and informal logic, the history of the relations between the notions of argument is partly similar to the history of the relations between the notions of algorithm. An intuitive broad notion is reduced to a narrow notion in order to be treated formally, but after some time the limitations induced by the narrow notion appear as too restrictive and scholars start considering the possibility of broadening it, even if the broader notion can only be partially formalized or cannot be made as precise as the narrow notion. The distinction between a narrow and a broad notion that has been presented in the case of algorithms has thus an analogy in the case of arguments. Firstly, the development of argumentation theory, and especially of informal logic as a reaction to the reduction of the notion of argument to logical consequence is similar to the criticism of the reduction of the notion of algorithm to the notion of a computable function. Secondly, several formal definitions of argument were developed in order to make the broader intuitive notion more precise, but after some time they were judged as insufficient to express human reasoning; analogously the notion of a function that is computable by a Turing machine has been recently perceived as too restrictive to express all the possibilities of human computation, although still considered as a good way to make the notion of algorithm precise. Thirdly, as in the case of algorithms, the broad notion precedes the narrow notion both historically and conceptually, even if the latter can be obtained by the definition of the former, if certain conditions are modified or abandoned. The similarities between algorithms and arguments do not concern only the history of their definitions. If one considers the relation between the two narrow notions of argument and algorithm and the relation between the two broad notions respectively, one might remark certain similarities. The attack made by argumentation theorists on premise (3), i.e. to the claim that arguments can be expressed as algorithms, was based on a comparison of the broader notion of argument with the narrow notion of algorithm. But if one now compares the broad notion of argument with the broad notion of algorithm, some similarities might need further investigation. Firstly, the broader notion of argument is not incompatible with a representation by means of diagrams, graphs, procedural forms, and other inferential schemes that can be expressed by algorithms. This is proved by the number of articles and results produced in AI by scholars who developed Toulmin's interpretation of an argument as a procedural form. Secondly, the attention devoted to pragmatics in argumentation theory is now emerging in computation theory too, especially in the development of algorithms that need to be interpreted by multi-agent systems, whose resources and background knowledge depend on the amount of interaction between the system and the environment and between the agents themselves. 200 Finally, the interest for the interpretation of the assertions of the interlocutor in the argumentative practice might be fruitfully compared to the interpretation of the information received from an agent in a complex system. The non-deterministic and indefinite aspects of the broader notion of algorithm might usefully be applied to the reconstruction of certain aspects of human argumentative practices. A deeper investigation of these and maybe other similarities between the broad notion of algorithm and the broad notion of argument might shed some light on a strictly foundational question that will be developed in the next paragraph: are there some specific features of human rationality that explain our argumentative practices and that cannot be reproduced by the mechanical computation of a multi-agent system? 8. Conclusion Argumentation theory was partly developed in the belief that there is much more to an argument than there is to an algorithm, but the broad notion of argument was compared with the narrow notion of algorithm. Along these lines one could develop a strategy to refute the AI-thesis, i.e. the claim that the argumentative reasoning of the human mind could be emulated by the computation of a machine. But if one considers a broader notion of algorithm, the AI-thesis might be raised anew: is there something in the broader notion of argument that cannot be captured by the broader notion of algorithm? This question might get a different answer based on recent developments in logic (non-monotonic logic, default logic, ...), in AI (multi-agent systems) and in computation theory (non-deterministic indefinite algorithms). If premise (3) of the argument introduced in § 2 cannot be easily refuted, one might ask oneself if the alternative strategies to refute the conclusion are still viable, after one has abandoned the implicit premise that an algorithm is a Turing-machine computable function. The claim that the argumentative reasoning of the mind can be emulated by a single machine was mainly a question concerning logic and the philosophy of mind, and not a question concerning argumentation theory, because the reasoning that was at stake there was neither dialectical nor dialogic, but rather a merely monologic calculus. Therefore it is possible to accept premise (3) and still deny the AI-thesis in its original formulation. In the Introduction to Argumentation in Artificial Intelligence, J. van Benthem apparently adopts this strategy when he reassures logicians, philosophers and argumentation theorists by saying that no AI theorist believes anymore that machines can emulate humans. Machines are rather useful to improve the understanding of human capacities: "Original visions of AI tended to emphasize hugely uninspiring, if terrifying, goals like machines emulating humans. [...] Understanding argumentation means understanding a crucial feature of ourselves, perhaps using machines to improve our performance, helping us humans be better at what we are" (Rahwan and Simari, 2009, p. viii). This is an easy move, but maybe not too convincing, for even if no AI scholar would claim anymore that a single machine could emulate the reasoning of a single human mind, she could still defend a variant of the AI-thesis reformulated in the light of recent developments of logic, computation theory, artificial intelligence, and argumentation theory: (ATAI-thesis) a multi-agent system can emulate the interactive reasoning of several human beings. 201 Recent developments of the applications of AI to argumentation theory suggest that several inter-subjective aspects of human argumentative interactions can be simulated by complex algorithms functioning on systems of interacting machines. It is no longer a question of how far the activities of the brain can be simulated by some physical device, but rather the question is why the application of AI to argumentation theory is so fruitful. For example, there is research on algorithms that produce new arguments, and successful implementations of argument-based machine learning. This paper does not aim to give a definite answer to the ATAI-question, but rather to show that the question is still open and cannot be easily liquidated as an obsolete or untenable claim. Once reformulated, the analysis of the ATAI-thesis (i.e. AI-thesis revisited in the light of Argumentation Theory) might have some effects on the foundation of argumentation theory itself, as we will claim in the following, after briefly mentioning what we mean here by foundational questions. According to our understanding, foundational problems in argumentation theory concern the creation of an adequate model that can be used to analyze argumentation practices: according to the reconstruction that we suggested elsewhere (Cantù & Testa 2006), such a foundational role might be played by the notions of dialectics, dialogue, intersubjectivity, pragmatics, but also by some ideal of argumentative rationality. Another relevant foundational issue might concern the bridging of the gap between different traditions (including formal and informal approaches to the reconstruction and evaluation of arguments) in order to provide a general framework for the development of argumentation studies. Now, the interaction between multi-agent systems is based on communication procedures that have strong similarities with the dialectical and dialogic interactions studied in argumentation theory, inasmuch as it is based on distributive cognition and on pragmatic elements as well as on syntactic and semantic aspects. So, the notions of dialectics, dialogue, intersubjectivity, and pragmatics play a major role also in the applications of artificial intelligence to argumentation theory. The ATAI-question asks if there are grounds for this similarity and implies that, if there are, then one should take the results of artificial intelligence into account when defining such concepts. Secondly, if mechanical computing can be considered as strictly argumentative, then the relevant features of argumentative rationality might already be captured by the algorithms of a multi-agent system: so, if one wants to claim that human argumentative practices contain some specificity ("the" rationality of argumentation), then one should exhibit some features (other than pragmatics and interaction) that could not be captured by the activity of some multi-agent system, and this, we believe, is a foundational task. Thirdly, the ATAI-thesis in connection with the distinction we suggested between a broad and a narrow notion of algorithm might suggest a new and fruitful way to bridge the gap between formal and informal approaches to argumentation theories, providing a new framework that could include both without misrepresenting their differences and peculiarities. Notes 1 The notion of a Turing-machine was first introduced by Alan Turing in 1937 in order to analyze the notion of computability. It is an ideal state machine made of an infinite one-dimensional tape divided into cells, each one able to contain one symbol, either '0' or '1'. The machine has a read-write head, which scans a single cell on the tape at a time, moving left and right along the tape to scan successive cells. The machine actions 202 are completely determined by the initial state of the machine, the symbols scanned by the head in the cells, and a list of instructions of the kind "if the machine is in the Initial State S0 and the current cell contains the Symbol y, then move into the Next State S1 taking Action z". 2 Cf. for example Toulmin 2001, p. 96, where the search for algorithms is criticized as a correlate of the search to ground objectivity in a unique methodological standpoint: "These arguments may leave mathematically-minded readers with a sense of loss. The dream of formal "algorithms" for guiding scientific procedures has a charm that will not quickly dissipate. For those who value mathematical exactitude above all other kinds of precision as the model for scientific inquiry, the alternative message of "different methods for different topics" will be a disappointment. Yet, over the centuries, we have been obliged to recognize a spectrum of different kinds of methods (in the plural) for sciences ranging from Newton's Planetary Theory-strictly factual and value-free, and in a style close to that of Euclid's Geometry-by way of empirical or functional sciences like geology, chemistry, physiology, and organic evolution, to those human sciences in which attempts to maintain value-neutrality finally proved vain." 3 The notion of conclusiveness, taken from Markov 1961, is similar to the notion of determinism, but might be fruitfully distinguished from the latter if one accepts Gurevich's characterization of non-deterministic algorithms as a special class of interactive arguments. "Imagine that you execute a non-deterministic algorithm A. In a given state, you may haperlve several alternatives for your action and you have to choose one of the available alternatives. The program of A tells you to make a choice but gives no instructions how to make the choice. [...] Whatever you do, you bring something external to the algorithm. In other words, it is the active environment that makes the choices." (Gurevich 2000, p. 25.) Gurevich's algorithm might be conclusive, because once the choice is made, the desired output might indeed be obtained, but it is non-deterministic, because depending on the choice there might be more than one sequence of steps leading from the input to the output. Besides, the algorithm might still be definite, at least in the sense that the arbitrariness does not depend on an ambiguous formulation of the algorithm, which allows for different interpretations, but rather on the introduction in the algorithm of something external to it. REFERENCES Blass, A., & Gurevich, Y. (2003). Algorithms: A quest for absolute definitions. Bulletin of European Association for Theoretical Computer Science, 81, 195-225. Cantù, P., & Testa I. (2006). Teorie dell'argomentazione [Theories of Argumentation], Milano: Bruno Mondadori. Chemla, K. (2005). The interplay between proof and algorithm in 3rd century China: The operation as prescription of computation and the operation as argument. In P. Mancosu, K. F. Jorgensen & S. A. Pedersen (Eds.), Visualization, Explanation and Reasoning Styles in Mathematics (pp. 123–145). Dordrecht: Springer. D'Agostini, F. (2010). Verità avvelenata. Buoni e cattivi argomenti nel dibattito pubblico [Poisoned truth. Good and bad arguments in the public debate] .Torino: Bollati Boringhieri. Eemeren, F.H. van, & Grootendorst, R. (2004). A Systematic Theory of Argumentation. The Pragma-Dialectical Approach. Cambridge: Cambridge University Press. 203 Gabbay, D.M., Johnson, R.H., Ohlbach, H.-J., & Woods, J. (Eds.) (2002). Handbook of the Logic of Argument and Inference. The Turn Towards the Practical. Amsterdam: Elsevier. Gödel, K. (1990). Kurt Gödel: Collected Works. Vol. II: Publications 1938-1974 (S. Feferman, J. W. Dawson , S. C. Kleene, G. H. Moore, R. M. Solovay & J. van Heijenoort, Eds.). New York: The Clarendon Press. Govier, T. (1987). Problems in Argument Analysis and Evaluation. Dordrecht: Foris. Gurevich, Y. (2000). Sequential abstract state machines capture sequential algorithms. ACM Transactions on Computational Logic, 1, 77–111. Johnson, R.H., & Blair, J.A. (2002). Informal logic and the reconfiguration of logic. In D.M. Gabbay, R.H. Johnson, H.-J. Ohlbach & J. Woods (Eds.), Handbook of the Logic of Argument and Inference. The Turn Towards the Practical (pp. 339396), Amsterdam: Elsevier. Knuth, D.E. (1997). The Art of Computer Programming. Vol. 1. Reading, MA: Addison-Wesley. Lorenzen, P., & Lorenz, K. (1978). Dialogische Logik. Darmstadt: Wissenschaftliche Buchgesellschaft. Markov, A.A. (1961). Theory of Algorithms. Jerusalem: The Israel Program for Scientific Translations. Perelman, C., & Olbrechts-Tyteca, L. (1958). Traité de l'argumentation. La nouvelle rhétorique. Paris: Presses Universitaires de France. Rahwan I., & Simari, G.R. (2009). Argumentation in Artificial Intelligence. Dordrecht: Springer. Scriven, M. (1980). The philosophical significance of informal logic. In J.A.Blair & R.H. Johnson (Eds.), Informal Logic: The First International Symposium (pp. 148–160). Inverness, CA: Edgepress. Sipser, M. (2006). Introduction to the Theory of Computation. Boston, MA: PWS. Toulmin, S. (2001). Return to Reason. Harvard: Harvard University Press. Turing, A.M. (1937). Computability and λ-Definability. The Journal of Symbolic Logic, 2: 153–163. Venn, J. (1881). Symbolic Logic. London: Macmillan & Co. Woods, J., Johnson, R.H., Gabbay, D., & Ohlbach, H.J. (2002). Logic and the practical turn. In D.M. Gabbay, R.H. Johnson, H.-J. Ohlbach & J. Woods (Eds.), Handbook of the Logic of Argument and Inference. The Turn Towards the Practical (pp. 1-40). Amsterdam: Elsevier. Woods, J. & Walton, D.N. (1982). Argument. The Logic of Fallacies. Toronto: Mc Graw-Hill.