 Third-but closely related to the second point-there is some empirical evidence which suggests that studies of lower quality have a tendency to over-estimate the efficacy of medical interventions (see footnote 7), and thus the use of QATs helps to accurately estimate the efficacy of medical interventions. In short, despite their low inter-rater and inter-tool reliability, QATs are an important component of medical research, and should be employed when performing a systematic review or meta-analysis. 5 Underdetermination of Evidential Significance The primary use of QATs is to estimate the quality of evidence from particular medical studies, and the primary use of such evidence is to estimate the strength (if any) of causal relations in relevant domains. The relata in these purported causal relations are, of course, the medical intervention under investigation and the change 17 in value of one or more parameters of a group of subjects. The best available QATs appropriate to a given domain differ substantially in the weight assigned to various methodological properties (§2), and thus generate discordant estimates of evidential quality when applied to the same evidence (§4). The differences between the best available QATs are fundamentally arbitrary. Although I assume that there must be a unique value (if at all) to the strength of purported causal relations in the domains in which these tools are employed, the low inter-tool reliability of QATs-together with the fundamentally arbitrary differences of their content-suggests that, in such domains and for such relations, there is no uniquely correct estimate of the quality of evidence. This is an instance of the general problem I call the underdetermination of evidential significance. Disagreement regarding the strength of evidence in particular scientific domains has been frequently documented with historical case studies. One virtue of examining the disagreement generated by the use of QATs is that such disagreements occur in highly controlled settings, are quantifiable using measures such as the κ statistic, and are about subjects of great importance. Such disagreements do not necessarily represent shortcoming on the part of the disagreeing scientists, and nor do such disagreements necessarily suggest a crude relativism. Two scientists who disagree about the strength of a particular piece of evidence can both be rational because their differing assessments of the strength of the same evidence can be due to their different weightings of fine-grained features of the methods which generated the evidence. This explains (at least in part) the low inter-rater and inter-tool reliability of QATs. Concluding that there is no uniquely correct
Third - but closely related to the second point - there is some empiric evidence which suggest that studies of lower timbre have a leaning to over - estimate the efficacy of aesculapian intervention (see footnote 7), and therefore the habit of QATs helps to accurately calculate the efficacy of medical interventions. In short, despite their low inter - rater and inter - tool dependability, QATs are an important part of medical inquiry, and should be hire when performing a systematic review or meta - psychoanalysis. 5 Underdetermination of Evidential Significance The primary use of QATs is to estimate the quality of evidence from particular aesculapian studies, and the elementary use of such evidence is to estimate the strength (if any) of causal sexual intercourse in relevant domains. The relata in these purported causal relations are, of course, the aesculapian interposition under investigation and the change 17 in value of one or more parameters of a group of subjects. The best available QATs appropriate to a given domain differ substantially in the system of weights assigned to various methodological place (§ 2), and therefore generate discordant estimates of evidential quality when applied to the same evidence (§ 4). The differences between the best available QATs are basically arbitrary. Although I assume that there must be a unique value (if at all) to the strength of purported causal relations in the domains in which these tools are employed, the low inter - tool reliability of QATs - together with the basically arbitrary differences of their subject - propose that, in such domains and for such relations, there is no uniquely correct estimate of the quality of evidence. This is an instance of the cosmopolitan problem I shout the underdetermination of evidential significance. Disagreement regard the strength of evidence in particular scientific sphere has been frequently documented with historic case study. One virtue of probe the disagreement generated by the manipulation of QATs is that such disagreements happen in highly controlled settings, are quantifiable using measurement such as the κ statistic, and are about subjects of great importance. Such disagreements do not inevitably represent shortcoming on the part of the disagreeing scientist, and nor do such disagreements necessarily suggest a crude relativism. Two scientists who disagree about the strength of a finical piece of evidence can both be rational because their differing assessments of the strength of the same evidence can be ascribable to their different weightings of very well - grained features of the methods which generated the evidence. This explains (at least in part) the abject inter - rater and inter - tool reliability of QATs. Concluding that there is no uniquely right
 Thlrd-but closely related tu the second ponbt-therx is soje empirkcal evidence which suggests tyat suldies of lower qualith have a nendency ro oter-estimate the xrficacy of meslcal nnverventions (see footnote 7), and thus the gsd lf QATs helps to accurately estimatq the egflcacy of medicwl imeervsntions. In short, despite their low inter-reter and inter-tpol reliability, QATs are aj imoortant component lf medical tssewech, and shouud be employed when petforming a systematic review or oeta-aualysis. 5 Uneeedehgrmination oh Evidvntial Signiflbance Tve primsry use of QATx iv ti estimate the qualitb of evidence from pwrticular mzdical studies, and thw primaty usa of wucf ebivende is ho xstimate ths strength (uf any) of causal rekaeppns in relevznt doiayns. The relata in these purported causan rslations are, of course, rhe medical interventlon under investigation and the change 17 in value of one or more oarcneterr ov a group of subjects. The best available QATs zpkroiriate to a given domain difgeg xobstantially iv the csifht assigned to vagious mgthodooogical pwopetties (§2), and thus generate diwcordant estpmatws of evidential qbality when cppliec to yhe same evidence (§4). The biffersnces betwefn the beag available QATs arv futdamentally arbitrary. Althjugh I aswume that tfere must fe a uniquf valma (if at all) to the streugth mf purportfd causal relations in the domamis in which tneve nools are emplpyed, the low ynter-tool relicbility jf QAGs-together with tie fundamentwlly arbitrarf differences of theiw cobtenr-suggesgr that, in such domains cud for suxh relations, there is ho uniquely corxtct estimate of thr qjalytj oh eviqance. This is an knsgsnce uf the geneval pronlem I call the undesdetsrmination of evidrnbial signuficance. Disagreement regarding the strtngth mf xvidenve yn particular scientific domaihs has befn nrequently dosumekted with histprical case studies. One virtue of examiiing the disagreement gwnerated by the use pf QATs is vhat stch disagseements occur in hithly controlled stttings, are quantifiabls usinc meadures such as the κ statistic, and are about subjects of great importance. Sych disagreements so npt nebesfcrily rqpredxnt shortcoming om the part of the disagreeing srientists, dnb nor do such disagreements nrcdssarily sugggst a crude relativism. Two scientixts who disagree about the stremgth of a particular piege of evieence can both ue rational because yheir bifferung asswssmcnts of the stfenbtm of the swme tvidznce can be due to their differenr weigytlngs of fine-graiied features uf thg methodf which generates ths evidgnce. This explsins (at least on part) the low integ-raeer and inter-tikl reliabilijy of QANs. Concluding that there is no uniquely correct
Third-but closely related to the second point-there empirical which suggests studies of lower over-estimate efficacy of medical (see footnote 7), thus the use of QATs helps accurately estimate the efficacy of medical interventions. In short, despite their low inter-rater inter-tool reliability, QATs are an important component of medical research, and should be when a review meta-analysis. 5 Underdetermination of Evidential Significance The primary use of QATs is to estimate the quality evidence from particular medical studies, and the primary of such evidence is estimate the strength (if any) causal in relevant The in purported causal relations of course, the medical intervention under investigation and the change 17 in value of one or more of a subjects. The available appropriate a given domain in the weight assigned to various and thus generate discordant estimates of evidential quality applied to same evidence (§4). The differences between best available QATs are fundamentally arbitrary. Although I that there must be a unique value (if at all) to the strength of purported in the domains in these tools are the inter-tool of with the arbitrary differences of their content-suggests that, in such domains and for relations, there is no uniquely correct estimate of the quality This an instance of general problem I call underdetermination evidential significance. Disagreement regarding of in has frequently with historical case studies. virtue of examining the disagreement by the use of occur in highly controlled settings, are quantifiable using such as the κ statistic, and are subjects of great importance. Such disagreements do not necessarily represent shortcoming on part of scientists, and nor do such disagreements necessarily suggest crude relativism. Two scientists disagree about the strength of a particular piece of can be rational their differing assessments the strength of same evidence can to their weightings fine-grained methods which generated the evidence. This (at in part) the low inter-rater inter-tool QATs. Concluding that there is no uniquely correct
 Third-but closely related to tHe second poInt-thEre Is sOmE empIricAl evidence whicH SuggEsts that studies of lower QualiTy HAve a TEnDency To over-eSTiMATe tHe EfFicAcY Of MedicAl iNtervenTions (see foOtnOtE 7), and thus the uSE oF QATs helps To aCcurately estImaTe the eFfIcaCY of meDicAl intErventIOns. In sHort, despiTe THeir loW Inter-raTER aNd inTer-tool reliabilitY, qAtS are an importanT compoNeNT oF MEdiCal Research, anD sHould BE employED wHEN PerFOrming a systemAtic review oR MetA-analySiS. 5 UnDErdeteRminaTiON of evidential SIgniFicance ThE primaRY use of Qats is to eStimatE thE quAlitY Of EvIdeNcE FroM PaRtiCUlaR medical StUdIes, anD the PRIMAry uSe oF sucH evidEnce is to estimAte The sTRenGth (if Any) of CausAl RelatIons in RelevAnT domains. The relaTa in These purpOrtEd CauSaL relaTIons arE, of CouRse, the mEdical iNTerVeNTIOn Under investigation AnD THe Change 17 in Value oF OnE oR More paraMeTerS of a GROup of SubjECtS. The best AvailaBLe qAts approPrIate to A gIveN doMain dIFfer SubstaNtially iN the wEIght assigned to VArious methodoLOgICAl PRopeRtiEs (§2), and thus geNeraTE disCordANt EstIMates Of eviDeNTiAL quality when applied To The samE evidEnce (§4). The differEnces betweEN THe best avAilaBLe qaTs are fundamenTally Arbitrary. ALThough I aSsume That therE must be a uNIQue value (If aT alL) to The STReNgth of purportED CausAl RelatioNs iN the domAinS in WhiCh tHeSe tools arE employeD, tHe LoW iNteR-tool REliabiliTy Of QaTS-toGetheR With thE fundAmenTaLlY ArbItrary dIFfEREnceS oF tHeir ConTeNt-sugGestS ThaT, in such Domains anD foR Such ReLaTions, thEre is no uniqueLy Correct estImAte Of the qUALity of evIdence. This is an instance oF The geneRal ProblEm I cAll the undErdEtermiNatIOn of evIdentiAl sigNiFicANCe. DisAGReEmeNt Regarding tHE StrEngth Of EvidEnce in pArticular scientifiC DomAins has been frEquEntlY DOcUmeNTeD WitH hIStoRICal case studies. ONe virtue of ExAMiNing the disAGreEmEnt geneRated by The usE Of QATs iS that such DisagreemEnTs ocCUR in Highly contRolled seTtings, are QUantiFIaBle usIng MeasurEs SucH as thE κ statIStiC, and aRe abouT sUbjectS of grEaT importaNce. Such disagreements do nOt neceSsariLy rEpresent sHorTComIng on the pArt oF the disagrEeiNg sCientIstS, And noR do sUCh DisAGreemEnts NEcessarilY SuGgeST A cRude relativISM. two ScienTisTS who diSagrEe about the strengtH Of a particular pIece OF EviDenCE can BoTh be rational beCauSe THEir diffeRiNg assessmenTs of the sTrENgth oF the saMe evidEnce can BE DuE To theiR difFerEnt weightIngS oF Fine-graInEd FEatureS of tHe MethodS which GEnerATEd the evidence. ThiS explAINs (at lEAst In parT) tHe low inTEr-raTer and inteR-tool reliabIlity oF QATS. ConcLuding tHaT there Is nO uNiquely corREct
 Third-but closely related  to the se condpoi nt- th ereis s ome empiricale vide nce which suggests tha t stu di e s of lo wer q ualityh av e a t en de ncy  t o  o ver-e sti mate th e efficacy  of  m edical inter v en tions (see  fo otnote 7), a ndthus t he  us e  of Q ATs  help s to a c curate ly estima te the ef f icacy o f me dica l interventions.I ns hort, despitetheirlo w  i n t er- rat er and int er -tool reliabi l it y , QAT s  are an impor tant compon e ntof med ic alr esearc h, an ds hou ld be emplo yedwhen perf orming a syste m atic re view o r m eta -ana l ys is . 5  U n der d et erm i nat ion of E vi de ntial  Sig n i f i canc e T he p rimar y use of QATs  is  toe sti matethe q uali ty  of e videnc e fro mparticular medi calstudies,and  t hepr imary use of  su chevidenc e is to est im a t e  t he strength (if an y) o fcausal r elatio n sin relevant  d oma ins. T he re lata in  these p urport e dca usal re la tionsar e,ofcours e , th e medi cal inte rvent i on under inves t igation and t h ec h an g e 17  in  value of o ne o r  mor e pa r am ete r s ofa gro up of subjects. The bestav ailabl e QAT s appropriate  to a give n d omain di ffer su b stantially inthe w eight assi g ned to v ariou s method ologicalp r operties  (§ 2),  an d t h u sgenerate disc o r dant  e stimate s o f evide nti alqua lit ywhen appl ied to t he  s am eevi dence (§4). Th edif fe ren ces b e tweenthe b estav ai l abl e QATsa re f unda me nt ally  ar bi trary . Al t hou gh I as sume that  th e re m us tbe a un ique value (i fat all) to  t hestreng t h  of purp orted causal relationsi n the d oma ins i n wh ich these  to ols ar e e m ployed , thelow i nt er- t o ol re l i ab ili ty  of QATs-t o g eth er wi th  the  fundam entally arbitraryd iff erences of th eir  con t e nt -su g ge s tsth a t,i n  such domains a nd for suc hr el ations, th e reis  no uni quely c orrec t  estima te of the  qualityof  evi d e nce . This isan insta nce of th e  gene r al  prob lem  I cal lthe  unde rdeter m ina tionof evi de ntialsigni fi cance. D isagreement regarding t he str ength  of  evidence  in par ticular s cien tific doma ins  ha s bee n f r equen tlyd oc ume n ted w ithh istorical ca ses t ud ies. One vi r t u e o f exa min i ng the  dis agreement generat e d by the use o f QA T s  is  th a t su ch  disagreements  oc cu r in highl ycontrolledsettings ,a re qu antifi able u sing me a s ur e s such  asthe  κ statis tic ,a nd areab ou t  subje ctsof  great  impor t ance . Such disagreemen ts do n ot ne c ess arily  r epresen t  sho rtcoming o n the partof the  dis agree ing sci en tists,  an dnor do suc h  disagree ments  necess ar ilysug gest a  cru d e  rela tivi sm . T wo scient i s ts wh od isa gree  abou tthestrengtho f a part icu l ar piec eofe v idence ca n both be ra tio nal b e c ause their diff e ri n g ass essmen ts ofthe str e ngt hof thesam e evidencecan be du e  to  t heir  differe nt  wei gh tin gs  of fine-graine d featur es of th e method s w h ichg enerat ed  the evidenc e. Th is explain s (at  leasti n p art ) t he l owinte r -rater and inter-tool re liab ility ofQATs.  Conc lu ding tha t  the re is no uniquely corr ect
 Third-but_closely related_to the second point-there_is some_empirical_evidence which_suggests_that studies of_lower quality have_a tendency to over-estimate_the efficacy of_medical_interventions (see footnote 7), and thus the use of QATs helps to accurately estimate_the_efficacy of_medical_interventions._In short, despite their low_inter-rater and inter-tool reliability, QATs_are an_important component of medical research, and should be_employed_when performing a_systematic review or meta-analysis. 5 Underdetermination of Evidential Significance_The primary use of QATs is_to estimate the_quality_of_evidence from particular medical_studies, and the primary use of_such evidence is to estimate the_strength (if any) of causal relations in_relevant domains. The relata in these_purported causal relations are, of_course, the_medical intervention under investigation and_the change 17_in value_of one or_more parameters of a group of_subjects. The best_available QATs appropriate to a given_domain_differ substantially in_the_weight_assigned to_various methodological properties_(§2),_and thus_generate_discordant estimates of evidential quality when_applied_to the same evidence (§4). The differences_between the best available_QATs_are fundamentally arbitrary. Although_I assume that there must_be a unique value (if at_all) to_the strength_of purported causal relations in the domains in which these tools_are employed, the low inter-tool reliability_of QATs-together with the_fundamentally arbitrary_differences_of their content-suggests_that,_in such_domains and for such relations, there is_no uniquely_correct estimate of the quality of_evidence. This is an_instance_of the general problem I call_the underdetermination of evidential significance. Disagreement_regarding the strength of evidence_in_particular_scientific domains has been frequently_documented with historical case studies. One_virtue of examining_the disagreement generated by the use of_QATs_is that such disagreements occur in_highly_controlled settings, are quantifiable using measures_such_as_the κ statistic, and are_about subjects of great importance. Such_disagreements do not necessarily represent shortcoming on the part_of the disagreeing_scientists, and nor do such_disagreements_necessarily_suggest a crude relativism. Two scientists who disagree about the_strength of_a particular piece_of evidence can both be rational because their differing assessments_of the strength of the same evidence_can be due to their different weightings of fine-grained features of_the methods which generated the evidence. This explains_(at least in part) the low inter-rater_and inter-tool reliability of_QATs._Concluding that there is no uniquely correct