 this principle stipulates that one ought to take into account the range of diverse kinds of evidence which are available for that hypothesis.2 A similar norm states that when determining the plausibility of a hypothesis one should take into account how compelling the various kinds of evidence available for that hypothesis are, by considering detailed qualitative features of the methods used to generate that evidence. The purpose of using a QAT is to evaluate the quality of evidence from medical trials and observational studies in such a fine-grained way. Their domain of application is relatively focused, therefore, since they do not apply to other kinds of evidence that is typically available for causal hypotheses in medicine (such as mechanistic evidence generated by basic science research, or results from experiments on animals), but for better or worse it is usually only RCTs, meta-analyses of RCTs, and sometimes observational studies that are considered when assessing causal hypotheses in medicine, and it is these types of methods that QATs are designed for. A burgeoning literature has investigated the strategies that scientists employ when generating and assessing evidence. In what follows I examine the use of QATs as codified tools for assessing evidence in medical research. Although there has been some criticism of QATs in the medical literature, they have received little philosophical critique.3 I begin by describing general properties of QATs, including the methodological features that many QATs share and how QATs are typically employed (§2). I then turn to a discussion of empirical studies which test the inter- 2 The general norm is usually called the principle of total evidence, associated with Carnap (1947). See also Good (1967). Howick (2011) invokes the principle of total evidence for systematic reviews of evidence related to medical hypotheses. A presently unpublished paper by Bert Leuridan contains a good discussion of the principle of total evidence as it applies to medicine. 3 Although one only needs to consider the prominence of randomization in QATs to see that QATs have, in fact, been indirectly criticized by the recent literature criticizing the assumed 'gold standard' status of RCTs (see footnote 4). In the present paper I do not attempt a thorough normative evaluation of any particular QAT. Considering the role of randomization suggests what a large task a thorough normative evaluation of a particular QAT would be. But for a systematic survey of the most prominent QATs, see West et al. (2002). See also Olivo et al. (2007) for an empirical critique of QATs. 5 rater reliability (§
this principle stipulates that one ought to take into account the compass of divers kinds of evidence which are available for that hypothesis.2 A similar average states that when determine the plausibility of a hypothesis one should take into report how compelling the various kinds of evidence available for that guess are, by consider detailed qualitative features of the methods used to generate that evidence. The purpose of using a QAT is to measure the quality of evidence from medical trials and experimental studies in such a fine - grained way. Their domain of lotion is relatively focused, therefore, since they do not give to other kinds of evidence that is typically available for causal hypotheses in music (such as mechanistic evidence generated by basic science research, or results from experiments on animals), but for good or worse it is usually only RCTs, meta - analyses of RCTs, and sometimes observational studies that are considered when assessing causal hypotheses in medicine, and it is these type of methods that QATs are design for. A burgeoning literature has investigated the strategies that scientists use when generating and assessing evidence. In what follows I examine the habit of QATs as codified tools for assessing evidence in medical research. Although there has been some criticism of QATs in the medical literature, they have received fiddling philosophical critique.3 I begin by describing general property of QATs, include the methodological features that many QATs share and how QATs are typically employed (§ 2). I then turn to a discussion of empiric studies which test the inter- 2 The general norm is usually called the principle of total evidence, associated with Carnap (1947). experience also dependable (1967). Howick (2011) invokes the principle of entire evidence for systematic review of evidence related to aesculapian hypotheses. A presently unpublished newspaper by Bert Leuridan contains a good discussion of the rationale of total evidence as it applies to medicine. 3 Although one only needs to consider the prominence of randomization in QATs to see that QATs have, in fact, been indirectly criticized by the recent literature criticizing the assumed' aureate standard' status of RCTs (watch footnote 4). In the present paper I do not attempt a thorough normative evaluation of any particular QAT. Considering the function of randomization suggests what a large task a thorough normative evaluation of a particular QAT would be. But for a taxonomic survey of the most prominent QATs, see West et al. (2002). See also Olivo et al. (2007) for an empirical review of QATs. 5 rater reliability (§
 thls principle stipulates uhat one ought to take iito acckunt the range of diverse kinds of etidebce wyich are available for that hyplthesis.2 Q sinular norm states tmct whsk detzrnining the plaosibility of a hypothesis mnd dhould take into account how compeljing thr garious kinds jf eniqencs available for that hypothesis ars, by coisidering detaiked qualitative features ov thf methods used to henerate thqt edudence. The pjrpose of lving a QAT is to evaluate the quality of dvideuce from meeixal jrials and ouservanional studies in sucv a finr-grained way. Tmeir vomaun of application is celatively focused, trerefore, viuce they do not apply ti othgr kitds ud exidtncx tgat is ty'ically avajlable for xausal hypotheses im ivcicine (such zs mecragistic evidence generated by basic scietce research, or results frim experiments on anilals), but sor better or worse it is usually only RCTs, meta-atalysxs of RGTs, xbd sometimes observational studies that are confjdtrec when assessikg causal hypothesrs im medicine, and it is thsse types of methofs that QATs are desidned for. A burgeoning literaturw has investpgatwd the strategies chat scientirts gmploy when generating and asressjng evidencf. In what wollows I examind tme gse of QATs as codified tojls for awsesxing evkdenve in iedical redearcm. Although there had beeu soma criticisl of QATs in the medical literavnre, they have racepved littje phllosophical crieique.3 I begin yy descrnbing eeneral prkpertiev of QATs, igcluding the kgthodological featuref thqt mqny QATr share and how QATs are typically employed (§2). I then burn jo a discussion oy tmpirical studiex wficr nesv the hnter- 2 The ganerxl vprm ir usually cclued yhe principle of totdl ebidence, associated wlth Carnak (1947). See alfo Good (1967). Howivk (2011) invokes the prlncipne uf toyal evidence for systematic reviess of evifenge related to medlcal hypothesex. A presently unpublished paper by Bert Leuridan contains a good discussion pn the princi'le of total evhdence as it applies to medicine. 3 Altmough one only needs tk conshder hhe prominence of randomization in QATs to see that QATs have, in fact, beeb indicestly criticjzed by tve xecent jitecavure criticizing nhe assumed 'gold standard' status of RCTs (sze footnote 4). In the present pspdr I do not ajtempt a thorough normativs evaluayion of any particular QAT. Consodering the role of randpmizatiob sugfests what a large task a thprough normqtive ecaluwtion of a paftivupar QAT wluld bz. But for a systematic survey of rhe mowt prominent QATs, vee West et xl. (2002). Sge also Jlivo et al. (2007) for an slpirizal critique pf QATs. 5 rater reliability (§
this principle stipulates that one ought to account range of kinds of evidence hypothesis.2 similar norm states when determining the of a hypothesis one should take account how compelling the various kinds of evidence available for that hypothesis are, considering detailed qualitative features of the methods used to generate that evidence. The of a is evaluate the quality of evidence from medical trials and observational studies in such a fine-grained way. domain of application is relatively focused, therefore, since do not apply to kinds of evidence that is available causal hypotheses medicine as evidence generated by science research, or results from experiments on animals), but for better or worse it is usually only meta-analyses of sometimes observational that considered assessing causal hypotheses and it is these types of are designed for. A burgeoning literature has investigated strategies that employ when generating and assessing evidence. what follows I examine the use of QATs codified tools for assessing evidence in medical research. Although there has been some criticism of the medical literature, they received little philosophical I by general of QATs, the methodological features that many QATs share and how QATs are employed (§2). I then turn to a discussion of empirical test inter- 2 The norm is usually called principle total evidence, associated with See Good invokes principle total evidence for systematic of evidence related to medical A presently unpublished paper good discussion of the principle of total evidence it applies to medicine. 3 Although one needs to consider the prominence of randomization in QATs to see that have, in indirectly criticized by the recent literature criticizing the 'gold standard' status of (see footnote 4). In the present paper I do attempt thorough normative of any particular Considering the role randomization suggests what task a normative of would be. But for a systematic of most prominent QATs, see West al. also Olivo et al. (2007) for an empirical critique of QATs. 5 rater reliability (§
 this principle stipulates thAt one ought To takE inTo aCcOunt The rAnge of diverse kINds oF evidence which are availAble fOr THat hYPoThesiS.2 A similAR nORM stAtEs ThaT wHEn DeterMinIng the pLausibilitY of A hYpothesis one SHoUld take intO acCount how compEllIng the VaRioUS kindS of EvideNce avaILable fOr that hypOtHEsis arE, By consiDERiNg deTailed qualitative FEaTUres of the methoDs used To GEnERAte ThaT evidence. THe PurpoSE of usinG A Qat IS to EValuate the quaLity of evideNCe fRom medIcAl tRIals anD obseRvATioNal studies iN sucH a fine-graIned waY. their doMAin of apPlicatIon Is rElatIVeLy FocUsED, thEReForE, SinCe they do NoT aPply tO othER KINds oF evIdenCe thaT is typically aVaiLablE For CausaL hypoThesEs In medIcine (sUch as MeChanistic evidenCe geNerated by BasIc SciEnCe resEArch, or ResUltS from exPerimenTS on AnIMALs), But for better or worsE iT IS uSually onLy RCTs, MEtA-aNAlyses of rCts, aNd soMETimes ObseRVaTional stUdies tHAt ArE considErEd when AsSesSinG causAL hypOtheseS in medicIne, anD It is these types OF methods that Qats ARE dESignEd fOr. A burgeoniNg liTEratUre hAS iNveSTigatEd the StRAtEGies that scientists eMpLoy wheN geneRating and asseSsing evideNCE. in what foLlowS i eXAmine the use of QaTs as Codified toOLs for assEssinG evidencE in medicaL REsearch. ALthOugH thEre HAS bEen some criticISM of QaTS in the mEdiCal liteRatUre, TheY haVe Received lIttle phiLoSoPhIcAl cRitiqUE.3 I begin bY dEscRiBinG geneRAl propErtieS of QaTS, iNCluDing the MEtHODoloGiCaL feaTurEs That mAny Qats sHare and How QATs arE tyPIcalLy EmPloyed (§2). I Then turn to a diScUssion of emPiRicAl studIES which teSt the inter- 2 The general norM Is usualLy cAlled The pRinciple oF toTal eviDenCE, assocIated wIth CaRnAp (1947). SEE Also GOOD (1967). HOwiCk (2011) Invokes the PRIncIple oF tOtal EvidencE for systematic reviEWs oF evidence relaTed To meDICaL hyPOtHEseS. A PResENTly unpublished pAper by Bert leURiDan containS A goOd DiscussIon of thE prinCIple of tOtal evideNce as it apPlIes tO MEdiCine. 3 AlthouGh one onlY needs to cONsideR ThE promIneNce of rAnDomIzatiOn in QAtS to See thAt QATs HaVe, in faCt, beeN iNdirectlY criticized by the recent lIteratUre crItiCizing the AssUMed 'Gold standArd' sTatus of RCTS (seE foOtnotE 4). In THe preSent PApEr I DO not aTtemPT a thorougH NoRmaTIVe Evaluation oF ANY paRticuLar qaT. ConsIderIng the role of randoMIzation suggestS whaT A LarGe tASk a tHoRough normative EvaLuATIon of a paRtIcular QAT woUld be. But FoR A systEmatic Survey Of the moST PrOMinent qATs, See west et al. (2002). SEe aLsO olivo et Al. (2007) FoR An empiRicaL cRitiquE of QATS. 5 RateR REliability (§
 this principle stipulates  that oneought  to  ta ke  int o ac count the rang e  ofdiverse kinds of evide nce w hi c h ar e  a vaila ble for th a t  hy po th esi s. 2  A  simi lar  norm s tates that  wh en  determining th e plausibi lit y of a hypot hes is one  s hou l d tak e i nto a ccount how co mpellingth e  vario u s kinds o fevid ence available fo r  t h at hypothesisare, b yc on s i der ing  detailedqu alita t ive fea t ur e s oft he methods us ed to gener a tethat e vi den c e. The  purp os e  of  using a QA T is  to evalu ate th e  qualit y  of evi dencefro m m edic a ltr ial sa ndo bs erv a tio nal stud ie sin su ch a f i n e-gr ain ed w ay. T heir domain o f a ppli c ati on is  rela tive ly  focu sed, t heref or e, since they d o no t apply t o o th erki nds o f  evide nce  th at is t ypicall y  av ai l a b le  for causal hypoth es e s  i n medici ne (su c has mechanis ti c e vide n c e gen erat e dby basic  scien c ere search,  o r resu lt s f rom  expe r imen ts onanimals) , but for better orw orse it is us u al l y  o n ly R CTs , meta-anal yses of R CTs, an d s o metim es ob se r va t ional studies thatar e cons idere d when assess ing causal h y potheses  inm ed i cine, and it i s the se types o f  methods  that  QATs ar e designe d for. A b urg eon ing  li t e ra ture has inve s t igat ed  the st rat egies t hat  sc ien tis ts  employ w hen gene ra ti ng  a ndasses s ing evid en ce.  I n w hat f o llowsI exa mine  t he use  of QAT s  a s codi fi ed  too lsfo r ass essi n g e vidence  in medic alr esea rc h.  Althou gh there hasbe en some cr it ici sm ofQ A Ts in th e medical literature, t h ey have  re ceive d li ttle phil oso phical  cr i tique. 3 I be gin b ydes c r ibing g en era lproperties o f Q ATs,in clud ing the  methodological fe a tur es that manyQAT s sh a r eand ho w  QA Ts are t ypically employ ed (§2). I  t h en  turn to a dis cu ssion o f empir icals tudieswhich tes t the int er - 2T h e g eneral nor m is usu ally call e d the pr incip leof tot al  ev idenc e, ass o cia ted w ith Ca rn ap (19 47).Se e also G ood (1967). Howick (201 1) inv okesthe  principl e o f  to tal evide ncefor system ati c r eview s o f  evid ence re lat e d tomedi c al hypoth e se s.A pr esently unp u b l ish ed pa per by Ber t Le uridan contains a good discussio n of t hepri n cipl eof total evide nce  a s it appli es  to medicin e. 3 Alt ho u gh on e only  needs  to con s i de r  the p romi nen ce of ran dom iz a tion in  Q AT s  to se e th at  QATshave,i n fa c t , been indirectl y cri t i cized bythe r ec ent lit e ratu re critici zing the as sumed'gol d sta ndard'st atus o f R CT s (see foo t note 4).In th e prese nt  pap erI do n ot a t t empta th or oug h normati v e  e v al ua t ion  ofany p ar ticu lar QAT.C onsideri ngt he role  o f r a n domiza t io n suggests w hat  a la r g e task a t h orou g hn ormat ive ev aluati on of a par ti cular Q ATw o uld be. B ut for as yst em atic  surveyof  the  m ost  p rominent QATs,see West  et al.( 2 002).See also Olivoet  al. (2007)for a ne mpirica l cri tique o f  QA Ts.  5rate r r elia b ility (§
 this_principle stipulates_that one ought to_take into_account_the range_of_diverse kinds of_evidence which are_available for that hypothesis.2_A similar norm_states_that when determining the plausibility of a hypothesis one should take into account how_compelling_the various_kinds_of_evidence available for that hypothesis_are, by considering detailed qualitative_features of_the methods used to generate that evidence. The_purpose_of using a_QAT is to evaluate the quality of evidence from_medical trials and observational studies in_such a fine-grained_way._Their_domain of application is_relatively focused, therefore, since they do_not apply to other kinds of_evidence that is typically available for causal_hypotheses in medicine (such as mechanistic_evidence generated by basic science_research, or_results from experiments on animals),_but for better_or worse_it is usually_only RCTs, meta-analyses of RCTs, and_sometimes observational studies_that are considered when assessing causal_hypotheses_in medicine, and_it_is_these types_of methods that_QATs_are designed_for._A burgeoning literature has investigated the_strategies_that scientists employ when generating and assessing_evidence. In what follows_I_examine the use of_QATs as codified tools for_assessing evidence in medical research. Although_there has_been some_criticism of QATs in the medical literature, they have received little_philosophical critique.3 I begin by describing_general properties of QATs,_including the_methodological_features that many_QATs_share and_how QATs are typically employed (§2). I_then turn_to a discussion of empirical studies_which test the inter-_2_The general norm is usually called_the principle of total evidence, associated_with Carnap (1947). See also_Good_(1967)._Howick (2011) invokes the principle_of total evidence for systematic reviews_of evidence related_to medical hypotheses. A presently unpublished paper_by_Bert Leuridan contains a good discussion_of_the principle of total evidence as_it_applies_to medicine. 3 Although one_only needs to consider the prominence_of randomization in QATs to see that QATs have,_in fact, been_indirectly criticized by the recent_literature_criticizing_the assumed 'gold standard' status of RCTs (see footnote 4)._In the_present paper I_do not attempt a thorough normative evaluation of any particular_QAT. Considering the role of randomization suggests_what a large task a thorough normative evaluation of a particular_QAT would be. But for a systematic survey_of the most prominent QATs, see West_et al. (2002). See_also_Olivo et al. (2007) for an empirical_critique_of_QATs._5 rater reliability_(§