 to take into account any 'defeating' properties of one's evidence. Suppose your evidence seems to provide definitive support for some 6 There are several commonly employed measures of effect size, including mean difference (for continuous variables), or odds ratio, risk ratio, or risk difference (for dichotomous variables). The weighting factor is sometimes determined by the QAT score, but a common method of determining the weight of a trial is simply based on the size of the trial (Egger, Smith, and Phillips, 1997), often by using the inverse variability of the data from a trial to measure that trial's weight (because inverse variability is correlated with trial size). 7 See, for example, Moher et al. (1998), Balk et al. (2002), and Hempel et al. (2011). 7 hypothesis, H1. But then you learn that there is a systematic error in the method which generated your evidence. Taking into account this systematic error, the evidence no longer supports H1 (perhaps instead the evidence supports a competitor hypothesis, H2). Had you not taken into account the fine-grained methodological information regarding the systematic error, you would have unwarranted belief in H1. You do not want to have unwarranted belief in a hypothesis, so you ought to take into account fine-grained methodological information. Here is a related argument: if one does not take into account all of one's evidence, including one's old evidence, then one is liable to commit the base-rate fallacy. In terms of Bayes' Theorem-p(H|e) = p(e|H)p(H)/p(e)-one commits the base-rate fallacy if one attempts to determine p(H|e) without taking into account p(H). Similarly, if one wants to determine p(H|e) then one ought to take into account the detailed methodological features which determine p(e|H) and p(e). One need not be a Bayesian to see the importance of assessing evidence at a fine-grain with QATs. For instance, Mayo's notion of'severe testing', broadly based on aspects of frequentist statistics, also requires taking into account fine-grained methodological details. The Severity Principle, to use Mayo's term, claims that "passing a test T (with e) counts as a good test of or good evidence for H just to the extent that H fits e and T is a severe test of H" (May
to take into account any' defeating' properties of one's evidence. presuppose your evidence seem to provide definitive support for some 6 There be several commonly hire meter of consequence size, including mean remainder (for continuous variables), or odds ratio, hazard ratio, or risk remainder (for dichotomous variables). The weighting component is sometimes determined by the QAT score, but a common method acting of determining the weight of a test is simply establish on the size of the trial (Egger, Smith, and Phillips, 1997), often by using the inverse variability of the data from a test to measure that trial's weight (because inverse variability is correlated with trial size). 7 See, for example, Moher et al. (1998), Balk et al. (2002), and Hempel et al. (2011). 7 hypothesis, H1. But then you learn that there is a systematic error in the method which generated your evidence. Taking into account this taxonomic mistake, the evidence no longer supports H1 (possibly alternatively the evidence defend a competitor hypothesis, H2). Had you not take into account the fine - grained methodological information involve the systematic error, you would have unwarranted belief in H1. You do not want to get unwarranted belief in a hypothesis, so you ought to take into account very well - grained methodological information. Here is a related argument: if one does not take into account all of one's evidence, include one's old evidence, then one is liable to commit the basis - rate fallacy. In terms of Bayes' Theorem - p(H|e) = p(e|H)p(H)/p(e)-one commits the base - pace fallacy if one attempts to determine p(H|e) without taking into account p(H). Similarly, if one wants to specify p(H|e) then one ought to take into history the detailed methodological features which specify p(e|H) and p(e). One need not be a Bayesian to see the importance of assess evidence at a fine - grain with QATs. For instance, Mayo's notion of'severe testing', broadly establish on aspects of frequentist statistic, also requires taking into account finely - ingrain methodological details. The Severity Principle, to use Mayo's terminus, claims that " passing a examination T (with e) counts as a beneficial test of or good evidence for H just to the extent that H fits e and T is a severe trial of H " (May
 to take into account any 'dtfeating' propertigs of onx's evidsnce. Supoose your evidence seems to 'rovude dtyinitive support for some 6 Thvre are swverel commonly employed measures kn effzcv size, includinn mean diffarence (for condivulus variables), or odds ratio, risk raeio, or tidk difference (sor cychofomous variables). The weighting facfor is vometimes detrrmined by the QAT score, bkt a common method of fetermining the qeight of a grial is spkply based on the size of the trial (Egger, Smitk, and Phillupw, 1997), lxten by usiig the inverse varlsbilitf of thr data from a brial to measure that trial's xeight (because inverfe variabhlnty is correlated wity rrial siza). 7 Sdw, fur txakpme, Mohfr xt al. (1998), Balk et al. (2002), and Hempel et al. (2011). 7 hypptrvxis, H1. But thsn you lqarn that there is a systematic error it tge method which generatwd your evidence. Takijg into ascount this systematic error, the evidence no longar su'purtw M1 (pefyaos instead the evidence supports a competitor gykotmesis, H2). Had you kot taken into accpujt jhe fine-grainea methodomogical informatioj regarqing rhe systeiativ error, you would have unwaeranted belivf ib H1. You do not wanc to have uncarranjed bekief in a hypothesis, so you ought to twke into zzcount fine-graindd kedhodological information. Hqre is a celatzd argumdnt: of one does not hake ltto account all of one's eeidence, influding one's old evidence, then one is liable jo womkit the yase-rabe fallacy. In tqrms of Bayes' Jheorem-p(H|z) = p(e|H)o(H)/p(e)-one cojmits tie base-rate sallacy if ona attempts to determige p(Y|e) wuthout gxking into accpunt p(H). Spmnlarly, if one wants to detevmine l(H|e) then one ouykt to take into avcojnt tje dqdailed methogoloeicxk feagures which aetetmine p(e|H) and p(e). One nees not be a Bayesiam bo see thg importagce of assessong evidence at a vine-gcain wmth QAYs. Sor instance, Mayo's notion of'sebere testlng', broadly basqd ok askects of frzquentist statistics, also requires takinj into account fine-graibed methodological beuails. The Sevxrity [rinciple, to use Mayo's term, coaims that "passinn a test T (with e) counfs as d goof test of or good evidence for H just to the extent that H fits e and T iw a setewe test of G" (Mau
to take into account any 'defeating' properties evidence. your evidence to provide definitive are commonly employed measures effect size, including difference (for continuous variables), or odds risk ratio, or risk difference (for dichotomous variables). The weighting factor is sometimes by the QAT score, but a common method of determining the weight of trial simply on size of the trial (Egger, Smith, and Phillips, 1997), often by using the inverse variability of data from a trial to measure that trial's (because inverse variability is with trial size). 7 See, example, et al. Balk al. and Hempel et (2011). 7 hypothesis, H1. But then you learn that there is a systematic error in the method generated your into account systematic the no longer supports instead the evidence supports a competitor you not taken into account the fine-grained methodological regarding the error, you would have unwarranted belief H1. You do not want to have unwarranted in a hypothesis, so you ought to take into account fine-grained methodological information. Here is argument: if one does take into account of evidence, one's evidence, then is liable to commit the base-rate fallacy. In terms of Bayes' = p(e|H)p(H)/p(e)-one commits the base-rate fallacy if one attempts to without into account p(H). if one wants to p(H|e) one ought to take the methodological p(e|H) p(e). need not be a to see the importance of evidence at a fine-grain notion of'severe testing', broadly based on aspects of statistics, also requires taking into account fine-grained details. The Severity Principle, to use Mayo's term, claims that "passing a T (with as a good test of or good evidence H just to the that H fits e and T is a severe of (May
 to take into account any 'defeaTing' properTies oF onE's eViDencE. SupPose your evidenCE seeMs to provide definitive sUpporT fOR somE 6 thEre arE severaL CoMMOnlY eMpLoyEd MEaSures Of eFfect siZe, includinG meAn Difference (foR CoNtinuous vaRiaBles), or odds raTio, Risk raTiO, or RIsk diFfeRence (For dicHOtomouS variableS). THE weighTIng factOR Is SomeTimes determined by THe qaT score, but a comMon metHoD Of DETerMinIng the weigHt Of a trIAl is simPLy BASEd oN The size of the tRial (Egger, SmITh, aNd PhilLiPs, 1997), oFTen by uSing tHe INveRse variabilIty oF the data fRom a trIAl to meaSUre that Trial's WeiGht (BecaUSe InVerSe VAriABiLitY Is cOrrelateD wItH triaL sizE). 7 sEE, For eXamPle, MOher eT al. (1998), Balk et al. (2002), anD HeMpel ET al. (2011). 7 HypotHesis, h1. But ThEn you Learn tHat thErE is a systematic eRror In the methOd wHiCh gEnErateD Your evIdeNce. taking iNto accoUNt tHiS SYStEmatic error, the evidEnCE No Longer suPports h1 (PeRhAPs insteaD tHe eVideNCE suppOrts A CoMpetitor HypothESiS, H2). had you nOt Taken iNtO acCouNt the FIne-gRained MethodolOgicaL Information regARding the systeMAtIC ErROr, yoU woUld have unwaRranTEd beLief IN H1. you DO not wAnt to HaVE uNWarranted belief in a hYpOthesiS, so yoU ought to take iNto account FINE-grained MethODoLOgical informatIon. HeRe is a relatED argumenT: if onE does not Take into aCCOunt all oF onE's eVidEncE, INcLuding one's old EVIdenCe, Then one Is lIable to ComMit The BasE-rAte fallacY. In terms Of baYeS' THeoRem-p(H|E) = P(e|H)p(H)/p(e)-oNe ComMiTs tHe basE-Rate faLlacy If onE aTtEMptS to deteRMiNE P(H|e) wItHoUt taKinG iNto acCounT P(H). SImilarlY, if one wanTs tO DeteRmInE p(H|e) theN one ought to taKe Into accounT tHe dEtaileD MEthodoloGical features which deterMIne p(e|H) aNd p(E). One nEed nOt be a BayeSiaN to see The IMportaNce of aSsessInG evIDEnce aT A FiNe-gRaIn with QATs. fOR inStancE, MAyo's Notion oF'severe testing', broaDLy bAsed on aspects Of fRequENTiSt sTAtIStiCs, ALso REQuires taking intO account fiNe-GRaIned methodOLogIcAl detaiLs. The SeVeritY principLe, to use MaYo's term, clAiMs thAT "PasSing a test T (With e) couNts as a gooD Test oF Or Good eVidEnce foR H JusT to thE extenT ThaT H fitS e and T Is A severE test Of h" (May
 to take into account any'defeating ' pro per tie sof o ne's  evidence. Sup p oseyour evidence seems to  prov id e  def i ni tivesupport fo r som e6The re ar e sev era l commo nly employ edme asures of ef f ec t size, in clu ding mean di ffe rence(f orc ontin uou s var iables ) , or o dds ratio ,r isk ra t io, orr i sk  dif ference (for dich o to m ous variables) . Thewe i gh t i ngfac tor is som et imesd etermin e db y the QAT score, bu t a commonm eth od ofde ter m iningthe w ei g htof a trialis s imply bas ed ont he size of thetrial(Eg ger , Sm i th ,and  P h ill i ps , 1 9 97) , oftenby  u singthei n v e rsevar iabi lityof the data f rom  a t r ial  to m easur e th at  tria l's we ight(b ecause inversevari ability i s c or rel at ed wi t h tria l s ize ). 7 Se e, fore xam pl e , Mo her et al. (1998),  B a l ket al. ( 2002), an dH empel et  a l.(201 1 ) . 7 h ypot h es is, H1.But th e nyo u learn  t hat th er e i s a  syst e mati c erro r in the  meth o d which genera t ed your evide n ce . Ta k ingint o account t hiss yste mati c  e rro r , the  evid en c en o longer supports H 1(perha ps in stead the evi dence supp o r t s a comp etit o rh ypothesis, H2) . Had  you not t a ken into  acco unt thefine-grai n e d method olo gic alinf o r ma tion regardin g thesy stemati c e rror, y ouwou ldhav eunwarrant ed belie fin  H 1.  Yo u don ot wantto  ha ve  un warra n ted be liefin a  h yp o the sis, so yo u ough tto  tak e i nt o acc ount fin e-grain ed method olo g ical  i nf ormatio n. Here is are lated argu me nt:  if on e does not  take into account allo f one's  ev idenc e, i ncludingone 's old  ev i dence,  thenone i slia b l e toc o mm itth e base-rat e fal lacy.  I n te rms ofBayes' Theorem-p(H | e)= p(e|H)p(H)/ p(e )-on e co mmi t st heba s e-r a t e fallacy if on e attempts  t o  d etermine p ( H|e )without  taking  into account  p(H). Si milarly,if  one w ant s to deter mine p(H |e) theno ne ou g ht  to t ake  intoac cou nt th e deta i led  meth odolog ic al fea tures  w hich det ermine p(e|H) and p(e).  One n eed n otbe a Baye sia n  to  see theimpo rtance ofass ess ing e vid e nce a t af in e-g r ain w ithQ ATs. Fori ns tan c e ,Mayo's noti o n of' sever e t e sting' , br oadly based on as p ects of freque ntis t sta tis t ics,  a lso requires t aki ng i nto acco un t fine-grai ned meth od o logic al det ails.The Sev e r it y  Princ iple , t o use May o's  t e rm, cla im st hat "p assi ng  a tes t T (w i th e ) counts as a good  test o f org ood  evid en ce forH  jus t to the e xtent thatH fits  e a nd Tis a se ve re tes t o fH" (May
 to_take into_account any 'defeating' properties_of one's_evidence._Suppose your_evidence_seems to provide_definitive support for_some 6 There are_several commonly employed_measures_of effect size, including mean difference (for continuous variables), or odds ratio, risk ratio,_or_risk difference_(for_dichotomous_variables). The weighting factor is_sometimes determined by the QAT_score, but_a common method of determining the weight of_a_trial is simply_based on the size of the trial (Egger, Smith,_and Phillips, 1997), often by using_the inverse variability_of_the_data from a trial_to measure that trial's weight (because_inverse variability is correlated with trial_size). 7 See, for example, Moher et_al. (1998), Balk et al. (2002),_and Hempel et al. (2011)._7 hypothesis,_H1. But then you learn_that there is_a systematic_error in the_method which generated your evidence. Taking_into account this_systematic error, the evidence no longer_supports_H1 (perhaps instead_the_evidence_supports a_competitor hypothesis, H2)._Had_you not_taken_into account the fine-grained methodological information_regarding_the systematic error, you would have unwarranted_belief in H1. You_do_not want to have_unwarranted belief in a hypothesis,_so you ought to take into_account fine-grained_methodological information._Here is a related argument: if one does not take into_account all of one's evidence, including_one's old evidence, then_one is_liable_to commit the_base-rate_fallacy. In_terms of Bayes' Theorem-p(H|e) = p(e|H)p(H)/p(e)-one commits_the base-rate_fallacy if one attempts to determine_p(H|e) without taking into_account_p(H). Similarly, if one wants to_determine p(H|e) then one ought to_take into account the detailed_methodological_features_which determine p(e|H) and p(e)._One need not be a Bayesian_to see the_importance of assessing evidence at a fine-grain_with_QATs. For instance, Mayo's notion of'severe_testing',_broadly based on aspects of frequentist_statistics,_also_requires taking into account fine-grained_methodological details. The Severity Principle, to_use Mayo's term, claims that "passing a test T_(with e) counts_as a good test of_or_good_evidence for H just to the extent that H fits_e and_T is a_severe test of H" (May