 multiple instruments have correlated measurements when applied to the same property being measured is referred to as inter-tool reliability. One QAT has inter-tool reliability with respect to another if its measurement of the quality of medical studies correlates with the measurement of the quality of the same studies by the other QAT. A QAT score is a measure on a relatively arbitrary scale, and the scales between multiple QATs are incommensurable, so constructs such as 'high quality' and 'low quality' are developed for each QAT which allow the results from different QATs to be compared. That is, when testing the inter-tool reliability of multiple QATs, what is usually being compared is the extent of their agreement regarding the categorization of particular medical trials into pre-defined bins of quality. Similar to assessments of inter-rater reliability, empirical evaluations of the inter-tool reliability have shown a wide disparity in the outcomes of multiple QATs when applied to the same primary-level studies; that is, the inter-tool reliability of QATs is poor. I should note, however, that there are few such assessments available, and those published thus far have varied with respect to the particular QATs assessed, the design of the reliability assessment, and the statistical analyses employed.10 An extensive investigation of inter-tool reliability was performed by Jüni and colleagues (1999). They amalgamated data from 17 studies which had tested a 10 For this latter reason I refrain from describing or illustrating the particular statistical analyses employed in tests of the inter-tool reliability of QATs, as I did in §3 on tests of the inter-rater reliability of QATs. Nearly every published test of inter-rater reliability uses a different statistic to measure agreement of quality assessment between tools. Analyses include Kendall's rank correlation coefficient (τ), Kendall's coefficient of concordance (W), and Spearman's rank correlation coefficient (ρ). 13 particular medical intervention (the use of low molecular weight heparin to prevent post-operative thrombosis), and they used 25 QATs to assess the quality of these 17 studies (thereby effectively performing 25 meta-analyses). The QATs that this group used were the same that Moher et al. (1995) had earlier described, which varied in the number of assessed study attributes, from a low of three attributes to a high of 34, and varied in the weight given to the various study attributes. Jüni and his colleagues noted that "most of these scoring systems
multiple instruments have correlated measurements when applied to the like place being measure is referred to as inter - tool dependability. One QAT has inter - tool dependability with respect to another if its measurement of the quality of aesculapian studies correlates with the measurement of the quality of the like study by the early QAT. A QAT score is a measure on a relatively arbitrary plate, and the scales between multiple QATs are incommensurable, so constructs such as' high quality' and' low timbre' are developed for each QAT which allow the results from unlike QATs to be compared. That is, when testing the inter - tool reliability of multiple QATs, what is normally being compared is the extent of their agreement regard the categorization of particular medical trial into pre - defined bins of quality. Similar to assessments of inter - rater dependability, empiric evaluation of the inter - tool reliability have shown a wide disparity in the outcomes of multiple QATs when applied to the same primary - degree studies; that is, the inter - tool reliability of QATs is inadequate. I should notice, however, that there be few such assessments available, and those published therefore far have varied with esteem to the particular QATs assessed, the invention of the reliability assessment, and the statistical analyses employed.10 An extensive investigation of inter - tool reliability was performed by Jüni and colleagues (1999). They amalgamated data from 17 studies which had test a 10 For this latter reason I abstain from trace or illustrating the particular statistical analyses employed in test of the inter - tool reliability of QATs, as I did in § 3 on tests of the inter - rater reliability of QATs. Nearly every published test of inter - rater reliability use a different statistic to quantify agreement of quality assessment between tools. Analyses include Kendall's rank and file correlation coefficient (τ), Kendall's coefficient of harmony (W), and Spearman's rank correlation coefficient coefficient (ρ). 13 particular medical intervention (the manipulation of low molecular weight unit heparin to prevent post - operative thrombosis), and they used 25 QATs to assess the quality of these 17 studies (thereby effectively performing 25 meta - analyses). The kat that this group used were the same that Moher et al. (1995) had sooner described, which varied in the number of assessed study attributes, from a low of three attributes to a high gear of 34, and varied in the weight given to the various study attributes. Jüni and his colleague note that " most of these marking system
 muptiple instruments have gorrelated measutenents xhen apllied to the same property being meadueed iw referred to as inter-gool reliwbility. Ine WQT has intxd-tool rcjiabjpity xith respect to another ix its measuremang lf the quality of medical studies cjrrelatrs with the measorememe of nht quality of the same studies by fhe othtr QAT. A QAT score is a measure on a relativfly wrbitrary scale, anf the scalew beeqeen multipld QATs are incommensurzble, so constructs such as 'high qualnty' and 'low qyalljy' are develiped sor each QAT which anlow thr results from dihferwnt QATs to be comparxd. That is, when testyng the ittzr-tool reliability of myltipne QDTs, dyat is uxuzlly bfinj compared js the extebt of their agreememt grgarding the categjryzation of particular medical trials inuo prs-defined bins of qualitt. Similar to assessmejts of ineer-rater reliability, empirical evaluations of the intec-tuol rcliacullty have shown a wide disparity in the outcomqa pf multiple QATs when applird tp the same prioary-lersl studies; that is, tje intet-tool eeliabiliuy of QATs is poor. I should note, however, than thwre are few such adsessments cvailanle, amd those published thus far have varief with reaoect to the partkcukas QATs assessed, the design of the rxliabnlity asresskent, agd the stahistigdl analyses employfd.10 An eftensive ijvestigation of inter-tool reliaumlity was pergosmeg by Jüni and golleagues (1999). Ther amalgamated bata froi 17 stjdies whicg had txsted a 10 For this latter sgason I reframn from dqscrubint or ilujstrating the larticulag ftatustical analyses ekpluged in tests of tht unter-tool relisbiuitr lf QWDs, as I did hn §3 un grsts uf the intev-rager teliability of QATs. Tearmy every published tcst of injer-rater weliability uxes a different stwtistmc to oeasute wgreement of quality assessmenf between tojls. Analyses incoude Kendall'x rank correlation coefficient (τ), Kendall'w coefficient of cincordance (W), and Spgarkan's rank cprrelwtion coexficient (ρ). 13 particulae medical intervektion (the use of low mkleculdr welght heparin to prevent post-operative thrombosis), and they used 25 QATs to awsess vhq quality or thrse 17 vtbdizs (therqby xfhectively performpng 25 meta-analyses). The QATs that vhis group ufed were the same that Moher eg al. (1995) had earuier described, which varjed in tne number of assessed study atttibutes, from a low of thvee attrivutes to a high of 34, and varied in the wzight tiven ti thc various studh aytvibutes. Jünl anv hns colleagues noted that "most of rhese wclring systems
multiple instruments have correlated measurements when applied same being measured referred to as inter-tool with respect to if its measurement the quality of medical studies correlates the measurement of the quality of the same studies by the other QAT. QAT score is a measure on a relatively arbitrary scale, and the scales multiple are so such as 'high quality' and 'low quality' are developed for each QAT which allow the results different QATs to be compared. That is, when the inter-tool reliability of QATs, what is usually being is extent of agreement the of particular medical into pre-defined bins of quality. Similar to assessments of inter-rater reliability, empirical evaluations of the inter-tool reliability shown a in the of QATs applied to the studies; that is, the inter-tool reliability poor. I should note, however, that there are such assessments and those published thus far have with respect to the particular QATs assessed, the of the reliability assessment, and the statistical analyses employed.10 An extensive investigation of inter-tool reliability by Jüni and colleagues They amalgamated data 17 which tested 10 For latter reason I refrain from describing or illustrating the particular statistical employed in tests of the inter-tool reliability of QATs, as in on tests of inter-rater reliability of QATs. every test of inter-rater reliability different to quality between Analyses include Kendall's rank coefficient (τ), Kendall's coefficient of (W), and Spearman's rank medical intervention (the use of low molecular weight to prevent post-operative thrombosis), and they used QATs to assess the quality of these 17 studies (thereby effectively performing meta-analyses). The this group used were the same that Moher al. (1995) had earlier which varied in the number of assessed study attributes, a of three to a high 34, and varied the weight given various study Jüni his "most of these scoring systems
 multiple instruments have coRrelated meAsureMenTs wHeN appLied To the same propeRTy beIng measured is referred tO as inTeR-Tool RElIabilIty. One Qat hAS IntEr-ToOl rElIAbIlity WitH respecT to another If iTs Measurement oF ThE quality of MedIcal studies cOrrElates WiTh tHE measUreMent oF the quALity of The same stUdIEs by thE Other QAt. a qAt scoRe is a measure on a reLAtIVely arbitrary sCale, anD tHE sCALes BetWeen multipLe qATs aRE incommENsURABle, SO constructs suCh as 'high quaLIty' And 'low QuAliTY' are deVelopEd FOr eAch QAT which AlloW the resulTs from DIfferenT qATs to bE compaRed. thaT is, wHEn TeStiNg THe iNTeR-toOL reLiabilitY oF mUltipLe QAtS, WHAt is UsuAlly Being Compared is the ExtEnt oF TheIr agrEemenT regArDing tHe cateGorizAtIon of particular MediCal trials IntO pRe-dEfIned bINs of quAliTy. SImilar tO assessMEntS oF INTeR-rater reliability, eMpIRIcAl evaluaTions oF ThE iNTer-tool rElIabIlitY HAve shOwn a WIdE dispariTy in thE OuTcOmes of mUlTiple QaTS whEn aPplieD To thE same pRimary-leVel stUDies; that is, the iNTer-tool reliabILiTY Of qaTs iS poOr. I should noTe, hoWEver, That THeRe aRE few sUch asSeSSmENts available, and thosE pUblishEd thuS far have varieD with respeCT TO the partIculAR Qats assessed, the dEsign Of the reliaBIlity assEssmeNt, and the StatisticAL Analyses EmpLoyEd.10 AN exTENsIve investigatION of iNtEr-tool rEliAbility Was PerForMed By jüni and coLleagues (1999). thEy AmAlGamAted dATa from 17 stUdIes WhIch Had teSTed a 10 FoR this LattEr ReASon i refraiN FrOM DescRiBiNg or IllUsTratiNg thE ParTicular StatisticAl aNAlysEs EmPloyed iN tests of the inTeR-tool reliaBiLitY of QATS, AS I did in §3 oN tests of the inter-rater reLIabilitY of qATs. NEarlY every pubLisHed tesT of INter-raTer relIabilItY usES A diffEREnT stAtIstic to meaSURe aGreemEnT of qUality aSsessment between toOLs. ANalyses includE KeNdalL'S RaNk cORrELatIoN CoeFFIcient (τ), Kendall's CoefficienT oF CoNcordance (W), ANd SPeArman's rAnk corrElatiON coeffiCient (ρ). 13 parTicular meDiCal iNTErvEntion (the uSe of low mOlecular wEIght hEPaRin to PreVent poSt-OpeRativE thromBOsiS), and tHey useD 25 QaTs to aSsess ThE quality Of these 17 studies (thereby efFectivEly peRfoRming 25 meta-AnaLYseS). The QATs tHat tHis group usEd wEre The saMe tHAt MohEr et AL. (1995) hAd eARlier DescRIbed, which VArIed IN ThE number of asSESSed Study AttRIbutes, From A low of three attribUTes to a high of 34, anD varIED in The WEighT gIven to the varioUs sTuDY AttributEs. jüni and his cOlleagueS nOTed thAt "most Of thesE scorinG SYsTEms
 multiple instruments have  correlate d mea sur eme nt s wh en a pplied to thes ameproperty being measure d isre f erre d  t o asinter-t o ol r eli ab il ity .O ne  QAThas  inter- tool relia bil it y with respe c tto another  if  its measure men t of t he  qu a lityofmedic al stu d ies co rrelateswi t h them easurem e n tof t he quality of the sa m e studies by t he oth er QA T .  AQAT  score isameasu r e on ar el a t i vel y  arbitrary sc ale, and th e  sc ales b et wee n  multi ple Q AT s  ar e incommens urab le, so co nstruc t s sucha s 'high  quali ty'  an d 'l o wqu ali ty '  ar e  d eve l ope d for ea ch  Q AT wh icha l l o w th e r esul ts fr om differentQAT s to becompa red.That  i s, wh en tes tingth e inter-tool re liab ility ofmul ti ple  Q ATs,w hat is  us ual ly bein g compa r edis t h eextent of their ag re e m en t regard ing th e  c at e gorizati on  of  par t i cular  med i ca l trials  intop re -d efinedbi ns ofqu ali ty.  Simi l ar t o asse ssmentsof in t er-rater relia b ility, empiri c al e va l uati ons  of the int er-t o ol r elia b il ity haveshown  a wi d e disparity in theou tcomes  of m ultiple QATswhen appli e d to the s amep ri m ary-level stud ies;that is, t h e inter- toolreliabil ity of QA T s  is poor . I  sh oul d n o t e,  however, tha t ther eare few  su ch asse ssm ent s a vai la ble, andthose pu bl is he dthu s far have var ie d w it h r espec t  to th e par ticu la rQ ATs  assess e d, t he d es ig n of  th erelia bili t y a ssessme nt, and t hes tati st ic al anal yses employed .1 0 An exten si veinvest i g ation of  inter-tool reliability was per for med b y Jü ni and co lle agues(19 9 9). Th ey ama lgama te d d a t a fro m 17  st ud ies whichh a d t ested  a  10For thi s latter reason Ir efr ain from desc rib ingo r  i llu s tr a tin gt hep a rticular statis tical anal ys e semployed i n  te st s of th e inter -tool reliabi lity of Q ATs, as I  d id i n §3on tests o f the in ter-rater relia b il ity o f Q ATs. N ea rly  ever y publ i she d tes t of i nt er-rat er re li abilityuses a different statis tic to  meas ure  agreemen t o f  qu ality ass essm ent betwee n t ool s. An aly s es in clud e  K end a ll'srank correlati o ncoe f f ic ient (τ), K e n d all 's co eff i cientof c oncordance (W), a n d Spearman's r ankc o rre lat i on c oe fficient (ρ).13pa r t icular m ed ical interv ention ( th e  useof low  molec ular we i g ht hepari n to  pr event pos t-o pe r ative t hr om b osis),  and  t hey us ed 25Q ATst o  assess the qual ity o f these 17studi es  (there b y ef fectivelyperforming25 met a-an alyse s). The  Q ATs th atth is group u s ed were t he sa me that  M oher  et  al. ( 1995 ) had e arli er  de scribed,w h ic h  v ar i edin t he nu mb er o f assesse d  study a ttr i butes,fr oma low of th r e e attribut esto ah i gh of 34,a nd v a ri e d inthe we ight g iven to the  v ariousstu d y  attribut es. Jünia ndhi s co lleagues  n oted  t hat  " most of these s coring s ystems
 multiple_instruments have_correlated measurements when applied_to the_same_property being_measured_is referred to_as inter-tool reliability._One QAT has inter-tool_reliability with respect_to_another if its measurement of the quality of medical studies correlates with the measurement_of_the quality_of_the_same studies by the other_QAT. A QAT score is_a measure_on a relatively arbitrary scale, and the scales_between_multiple QATs are_incommensurable, so constructs such as 'high quality' and 'low_quality' are developed for each QAT_which allow the_results_from_different QATs to be_compared. That is, when testing the_inter-tool reliability of multiple QATs, what_is usually being compared is the extent_of their agreement regarding the categorization_of particular medical trials into_pre-defined bins_of quality. Similar to assessments_of inter-rater reliability,_empirical evaluations_of the inter-tool_reliability have shown a wide disparity_in the outcomes_of multiple QATs when applied to_the_same primary-level studies;_that_is,_the inter-tool_reliability of QATs_is_poor. I_should_note, however, that there are few_such_assessments available, and those published thus far_have varied with respect_to_the particular QATs assessed,_the design of the reliability_assessment, and the statistical analyses employed.10_An extensive_investigation of_inter-tool reliability was performed by Jüni and colleagues (1999). They amalgamated_data from 17 studies which had_tested a 10 For_this latter_reason_I refrain from_describing_or illustrating_the particular statistical analyses employed in tests_of the_inter-tool reliability of QATs, as I_did in §3 on_tests_of the inter-rater reliability of QATs._Nearly every published test of inter-rater_reliability uses a different statistic_to_measure_agreement of quality assessment between_tools. Analyses include Kendall's rank correlation_coefficient (τ), Kendall's_coefficient of concordance (W), and Spearman's rank_correlation_coefficient (ρ). 13 particular medical intervention_(the_use of low molecular weight heparin_to_prevent_post-operative thrombosis), and they used_25 QATs to assess the quality_of these 17 studies (thereby effectively performing 25 meta-analyses)._The QATs that_this group used were the_same_that_Moher et al. (1995) had earlier described, which varied in_the number_of assessed study_attributes, from a low of three attributes to a high_of 34, and varied in the weight_given to the various study attributes. Jüni and his colleagues noted_that "most of these scoring systems