 allow us to identify a uniquely best QAT. There is a point at which one can no longer argue against philosophical optimism. The underdetermination of evidential significance is a hard problem; like other hard philosophical problems, it does not preclude optimism. One could put aside the aim of finding a principled basis for selecting among the available QATs, and instead perform a selection based on their historical performance. Call this a 'naturalist' selection of QATs.13 Since QATs are employed to estimate the quality of evidence from medical studies, and such evidence is used to estimate the strength of causal relations, the naturalist approach would involve selecting QATs based on a parameter determined by the 'fit' between (i) the strength of presently known causal relations and (ii) the quality of the evidence for such causal relations available at a particular time, as determined in retrospect by currently available QATs. The best QAT would be the one with the best average fit between (i) and (ii). Such an assessment of QATs would be of some value. It would be fundamentally limited, though, given an epistemic circularity. In the domains in which QATs are employed, the best epistemic access to the strength of causal relations is the total evidence from all the available medical studies, summarized by a careful systematic review (which, in this domain, usually takes the form of a metaanalysis), appropriately weighted to take into account relevant methodological features of those studies. But of course, those very weightings are generated by QATs. The naturalist approach to assessing QATs, then, itself requires the employment of QATs. 13 Such an approach was first suggested to me by Jim Tabery. 20 The underdetermination of evidential significance is not the same problem that is often associated with Duhem and Quine. One formulation of the standard underdetermination problem-underdetermination of theory by evidence-holds that there are multiple theories compatible with a given body of evidence. The underdetermination of evidential significance is the prior problem of settling on the strength of a given piece of evidence in the first place. Indeed, one may wish to say that an appropriate name for the present problem is just the inverse of the Quinean locution: underdetermination of evidence by theory. Our best theories of inference underdetermine the strength of evidence, exemplified by tools such as QATs. 6 QATs and Hierarchies The most frequently used tools for assessing the quality of medical studies are not
allow us to identify a uniquely best QAT. There be a item at which one can no longer argue against philosophical optimism. The underdetermination of evidential significance is a hard trouble; like other hard philosophical problem, it does not preclude optimism. One could put apart the aim of finding a principled footing for selecting among the available QATs, and instead perform a survival based on their historical performance. Call this a' naturalist' selection of QATs.13 Since kat are employed to estimate the quality of evidence from aesculapian studies, and such evidence is used to estimate the strength of causal relations, the naturalist access would involve selecting QATs based on a parameter determined by the' paroxysm' between (i) the strength of presently known causal relations and (ii) the quality of the evidence for such causal relations available at a particular time, as determined in retrospect by currently available QATs. The best QAT would be the one with the best average fit between (i) and (ii). Such an assessment of QATs would be of some value. It would be basically circumscribed, though, given an epistemic circularity. In the domains in which QATs are use, the best epistemic access to the strength of causal sexual intercourse is the entire evidence from all the available medical studies, sum up by a careful systematic review (which, in this domain, usually takes the form of a metaanalysis), appropriately slant to take into account relevant methodological features of those studies. But of path, those very weight are generate by QATs. The naturalist approach to assessing QATs, then, itself requires the use of QATs. 13 Such an approach was first suggested to me by Jim Tabery. 20 The underdetermination of evidential significance is not the same problem that is often associated with Duhem and Quine. One formulation of the standard underdetermination trouble - underdetermination of theory by evidence - holds that there are multiple theories compatible with a given torso of evidence. The underdetermination of evidential significance is the anterior problem of settling on the strength of a given firearm of evidence in the first place. Indeed, one may wish to read that an appropriate name for the present problem is merely the inverse of the Quinean locution: underdetermination of evidence by hypothesis. Our best theories of inference underdetermine the strength of evidence, exemplified by cock such as QATs. 6 kat and Hierarchies The most frequently practice tools for assessing the quality of medical studies are not
 alpow us to identify a uniduely best QAT. Jhwre is a poiht at whkch one can no longer argue egaibst pyilosophical optimism. Ghe underfeterminqtioi of evidential significance ia a hcrv problem; like pther hard philosophical pfoylems, it does not preclude optimism. Jne coukd put aside the aim jf fjnding a principled basis for seledting akong the avaikable QATs, and instead pervorm a selection based on their hustowucal performxnce. Call uhns a 'naturamist' selection of QATs.13 Since QAGs arz employed jo edjimate the qnality of evidence from megical syudies, and sucm evivencw is used to estimate the strength of cwusal reldtnons, the naturalist appeoach wound ivcolxe aekedting EATa based on a parametee determined by the 'fyn' between (i) tge strqndth of presently known causal relations ans (ii) the quality of the evidence for such caosal relatyons available at a particular time, as determined in rxtfoskegb by xugrently available QATs. The best QAT would be ege oke with the best average fiy heyreen (i) and (ii). Such ch zssessment of QATs would fe of some valte. Iy would be fundamentally linited, though, tiven an epistemic circularitv. In tne dokains in which QATs are empmoyed, the bfst epistsoic access to thd snrencth of causal relations is the totao evndence ffom sll thq availablf medlwal studies, summarlzed yy a wareful sydtematic review (which, in this domain, usually jakas nhe form jf a ketaanalysis), wppropriately ceighted to txke into abcount renevant methjdological fedjures of thosx studies. But of xourse, gfose very weigntings arv yenerated by QATs. The naturelisj zpproach to asszrwing QATs, then, otsdlf rvqumres eve employmend of QAGx. 13 Suzh an ak'roccf wax first suggested to me gy Jim Tabery. 20 The ukderdeternination of evidentiak significance is jot tie samx probkem that is often associated with Duhem anf Qmine. One formtlatlon jf the staudard underdetermination problem-underdetxrmination of theory bi evidence-holds that jheve are multi'le thqories cokpatible with a giveb body of evidenct. The underdeterminatioh of eeidenhial significance is the prior problem of settling on the strength of a guven pmese of evidehce on thv fnrsc place. Indxev, one may wish to say that an appropriate name fur the psefent problem is just the invrrre of the Quivean locution: underdeterjination of evidence by theory. Our best theories of inference ukderdeternine fhe stggngvh of evidence, exempkified by tiols suxh ad QATs. 6 QATs xnd Hlerarchies The mkst frequently used tools for aswessint hhe quality of mxdical studics are hot
allow us to identify a uniquely best is point at one can no The of evidential significance a hard problem; other hard philosophical problems, it does preclude optimism. One could put aside the aim of finding a principled basis selecting among the available QATs, and instead perform a selection based on their performance. this 'naturalist' of QATs.13 Since QATs are employed to estimate the quality of evidence from medical studies, and evidence is used to estimate the strength of relations, the naturalist approach involve selecting QATs based on parameter by the between the of presently known relations and (ii) the quality of the evidence for such causal relations available at a particular time, determined in currently available The QAT be the one best average fit between (i) and assessment of QATs would be of some value. would be limited, though, given an epistemic circularity. the domains in which QATs are employed, the epistemic access to the strength of causal relations is the total evidence from all the studies, summarized by a systematic review (which, this usually the of a appropriately weighted to take into account relevant methodological features of those But of course, those very weightings are generated by QATs. approach assessing QATs, then, requires the employment of 13 an approach was first me Jim underdetermination evidential is not the same that is often associated with and Quine. One formulation of theory by evidence-holds that there are multiple compatible with a given body of evidence. underdetermination of evidential significance is the prior problem of settling on the of a of evidence in the first place. Indeed, one wish to say that appropriate name for the present problem is just the of Quinean locution: of evidence by Our best theories inference underdetermine the evidence, exemplified tools as and Hierarchies The most frequently used for the quality of medical studies not
 allow us to identify a uniquelY best QAT. ThEre is A poInt At WhicH one Can no longer argUE agaInst philosophical optimIsm. ThE uNDerdETeRminaTion of eVIdENTiaL sIgNifIcANcE is a hArd Problem; Like other hArd PhIlosophical pRObLems, it does Not Preclude optiMisM. One coUlD puT Aside The Aim of FindinG A princIpled basiS fOR selecTIng amonG THe AvaiLable QATs, and insteAD pERform a selectioN based On THeIR HisTorIcal perforMaNce. CaLL this a 'nATuRALIst' SElection of QATS.13 Since QATs aRE emPloyed To EstIMate thE qualItY Of eVidence from MediCal studieS, and suCH evidenCE is used To estiMatE thE strENgTh Of cAuSAl rELaTioNS, thE naturalIsT aPproaCh woULD INvolVe sElecTing QaTs based on a paRamEter DEteRmineD by thE 'fit' BeTween (I) the stRengtH oF presently known CausAl relatioNs aNd (Ii) tHe QualiTY of the EviDenCe for suCh causaL RelAtIONS aVailable at a particuLaR TImE, as deterMined iN ReTrOSpect by cUrRenTly aVAIlablE QATS. thE best QAT Would bE ThE oNe with tHe Best avErAge Fit BetweEN (i) anD (ii). SucH an assesSment OF QATs would be of SOme value. It wouLD bE FUnDAmenTalLy limited, thOugh, GIven An epIStEmiC CircuLaritY. IN ThE Domains in which QATs aRe EmployEd, the Best epistemic Access to thE STRength of CausAL rELations is the toTal evIdence from ALl the avaIlablE medical Studies, suMMArized by A caRefUl sYstEMAtIc review (which, IN This DoMain, usuAllY takes tHe fOrm Of a MetAaNalysis), apPropriatElY wEiGhTed To takE Into accoUnT reLeVanT methODologiCal feAturEs Of THosE studieS. buT OF couRsE, tHose VerY wEightIngs ARe gEnerateD by QATs. ThE naTUralIsT aPproach To assessing QAts, Then, itself ReQuiRes the EMPloyment Of QATs. 13 Such an approach was FIrst sugGesTed to Me by jim Tabery. 20 the UnderdEteRMinatiOn of evIdentIaL siGNIficaNCE iS noT tHe same probLEM thAt is oFtEn asSociateD with Duhem and Quine. oNe fOrmulation of tHe sTandARD uNdeRDeTErmInATioN PRoblem-underdeteRmination oF tHEoRy by evidenCE-hoLdS that thEre are mUltipLE theoriEs compatiBle with a gIvEn boDY Of eVidence. The UnderdetErminatioN Of eviDEnTial sIgnIficanCe Is tHe priOr probLEm oF settLing on ThE strenGth of A gIven piecE of evidence in the first plAce. IndEed, onE maY wish to saY thAT an AppropriaTe naMe for the prEseNt pRobleM is JUst thE invERsE of THe QuiNean LOcution: unDErDetERMiNation of eviDENCe bY theoRy. OUR best tHeorIes of inference undERdetermine the sTrenGTH of EviDEnce, ExEmplified by tooLs sUcH AS QATs. 6 QATS aNd HierarchiEs The mosT fREquenTly useD tools For asseSSInG The quaLity Of mEdical stuDieS aRE not
 allow us to identify a un iquely bes t QAT . T her eis a  poi nt at which on e  can  no longer argue again st ph il o soph i ca l opt imism.T he u nde rd et erm in a ti on of  ev identia l signific anc eis a hard pr o bl em; like o the r hard philo sop hicalpr obl e ms, i t d oes n ot pre c lude o ptimism.On e  could put asi d e  t he a im of finding a p r in c ipled basis fo r sele ct i ng a mon g t he availab le  QATs ,  and in s te a d per f orm a selecti on based on the ir his to ric a l perf orman ce .  Ca ll this a ' natu ralist' s electi o n of QA T s.13 Si nce QA Tsare  emp l oy ed  to  e s tim a te  th e  qu ality of  e vi dence  fro m m e dica l s tudi es, a nd such evide nce  isu sed  to e stima te t he  stre ngth o f cau sa l relations, th e na turalistapp ro ach  w ouldi nvolve  se lec ting QA Ts base d  on  a p a ra meter determined b yt h e'fit' be tween( i)  t h e streng th  of  pre s e ntlyknow n  c ausal re lation s  a nd  (ii) t he  quali ty  of  th e evi d ence  for s uch caus al re l ations availab l e at a partic u la r ti m e, a s d etermined i n re t rosp ectb ycur r ently  avai la b le QATs. The best QATwo uld be  theone with thebest avera g e fit betw een( i) and (ii). Such  an a ssessmento f QATs w ouldbe of so me value. I t wouldbefun dam ent a l ly  limited, tho u g h, g iv en an e pis temic c irc ula rit y.In  the doma ins in w hi ch  Q AT s a re em p loyed, t he  be st  ep istem i c acce ss to  the  s tr e ngt h of ca u sa l rela ti on s is  th etotal  evi d enc e fromall the a vai l able  m ed ical st udies, summar iz ed by a ca re ful  syste m a tic revi ew (which, in this doma i n, usua lly  take s th e form of  ametaan aly s is), a ppropr iatel ywei g h ted t o ta kein to account r ele vantme thod ologica l features of thos e  st udies. But of  co urse , th ose ve r y w ei g hti n g s are generated  by QATs.Th e  n aturalista ppr oa ch to a ssessin g QAT s , then,  itself r equires t he  emp l o yme nt of QATs . 13 Suc h an appr o ach w a sfirst  su ggeste dtome by  Jim T a ber y. 20  The u nd erdete rmina ti on of ev idential significance i s notthe s ame  problemtha t  is  often as soci ated withDuh emand Q uin e . One  for m ul ati o n ofthes tandard u n de rde t e rm ination pro b l e m-u nderd ete r minati on o f theory by evide n ce-holds thatther e are  mu l tipl etheories compa tib le w ith a gi ve n body of e vidence.  T h e und erdete rminat ion ofe v id e ntialsign ifi cance isthe  p r ior pro bl em of set tlin gon the  stren g th o f a given piece of  evid e n ce in the  firs tplace.I ndee d, one may  wish to sa y that  anappro priatena me for  th epresent pr o blem is j ust t he inve rs e of  th e Quin eanl o cutio n: u nd erd eterminat i o no fev i den ce b y the or y. O ur best t h eories o f i n ference  u nde r d etermi n et h e strength  of  evid e n ce, exempl i fied by tools  suchas QAT s. 6 QA T s a nd  Hierar chi e s  The most  frequent l y u se d to ols foras sess in g t he  quality of med ical stu dies are n ot
 allow_us to_identify a uniquely best_QAT. There_is_a point_at_which one can_no longer argue_against philosophical optimism. The_underdetermination of evidential_significance_is a hard problem; like other hard philosophical problems, it does not preclude optimism._One_could put_aside_the_aim of finding a principled_basis for selecting among the_available QATs,_and instead perform a selection based on their_historical_performance. Call this_a 'naturalist' selection of QATs.13 Since QATs are employed_to estimate the quality of evidence_from medical studies,_and_such_evidence is used to_estimate the strength of causal relations,_the naturalist approach would involve selecting_QATs based on a parameter determined by_the 'fit' between (i) the strength_of presently known causal relations_and (ii)_the quality of the evidence_for such causal_relations available_at a particular_time, as determined in retrospect by_currently available QATs._The best QAT would be the_one_with the best_average_fit_between (i)_and (ii). Such_an_assessment of_QATs_would be of some value. It_would_be fundamentally limited, though, given an epistemic_circularity. In the domains_in_which QATs are employed,_the best epistemic access to_the strength of causal relations is_the total_evidence from_all the available medical studies, summarized by a careful systematic review_(which, in this domain, usually takes_the form of a_metaanalysis), appropriately_weighted_to take into_account_relevant methodological_features of those studies. But of course,_those very_weightings are generated by QATs. The_naturalist approach to assessing_QATs,_then, itself requires the employment of_QATs. 13 Such an approach was_first suggested to me by_Jim_Tabery._20 The underdetermination of evidential_significance is not the same problem_that is often_associated with Duhem and Quine. One formulation_of_the standard underdetermination problem-underdetermination of theory_by_evidence-holds that there are multiple theories_compatible_with_a given body of evidence._The underdetermination of evidential significance is_the prior problem of settling on the strength of_a given piece_of evidence in the first_place._Indeed,_one may wish to say that an appropriate name for_the present_problem is just_the inverse of the Quinean locution: underdetermination of evidence by_theory. Our best theories of inference underdetermine_the strength of evidence, exemplified by tools such as QATs. 6_QATs and Hierarchies The most frequently used tools_for assessing the quality of medical studies_are not