title: Evaluation of the Quality of Prognosis Studies in Systematic Reviews Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) Checklist Health related quality of life in the first year after diagnosis in children with brain tumours compared with matched healthy controls; a prospective longitudinal study. A prospective study of anxiety, depression, and behavioral changes in the first year after a diagnosis of childhood acute lymphoblastic leukemia “Vitality” a Missing Link in Adjustment to Childhood Cancer Hypnosis and nonhypnotic techniques for reduction of pain and anxiety during painful procedures in children and adolescents with cancer. A Pilot Exploration of Symptom Trajectories in Adolescents With Cancer During Chemotherapy
abstract: Prognosis studies are investigations of future events or the evaluation of associations between risk factors and health outcomes in population s of patients ( 1 ) . The results of such studies improve our underst and ing of the clinical course of a disease and assist clinicians in making informed decisions about how best to manage patients . Prognostic research also informs the design of intervention studies by helping define subgroups of patients who may benefit from a new treatment and by providing necessary information about the natural history of a disorder ( 2 ) . There has recently been a rapid increase in the use of systematic review methods to synthesize the evidence on research questions related to prognosis . It is essential that investigators conducting systematic review s thoroughly appraise the method ologic quality of included studies to be confident that a study 's design , conduct , analysis , and interpretation have adequately reduced the opportunity for bias ( 3 , 4 ) . Caution is warranted , however , because inclusion of method ologically weak studies can threaten the internal validity of a systematic review ( 4 ) . This follows abundant empirical evidence that inadequate attention to biases can cause invalid results and inferences ( 5 - 9 ) . However , there is limited consensus on how to appraise the quality of prognosis studies ( 1 ) . A useful framework to assess bias in such studies follows the basic principles of epidemiologic research ( 10 , 11 ) . We focus on 6 areas of potential bias : study participation , study attrition , prognostic factor measurement , confounding measurement and account , outcome measurement , and analysis . The main objectives of our review of review s are to describe methods used to assess the quality of prognosis studies and to describe how well current practice s assess potential biases . Our secondary objective is to develop recommendations to guide future quality appraisal , both within single studies of prognostic factors and within systematic review s of the evidence . We hope this work facilitates future discussion and research on biases in prognosis studies and systematic review s. Methods Literature Search and Study Selection We identified systematic review s of prognosis studies by search ing MEDLINE ( 1966 to October 2005 ) using the search strategy recommended by McKibbon and colleagues ( 12 ) . This strategy combines broad search terms for systematic review s ( systematic review .mp ; meta- analysis .mp ) and a sensitive search strategy for prognosis studies ( cohort , incidence , mortality , follow-up studies , prognos * , predict * , or course ) . We also search ed the reference lists of included review s and method ologic papers to identify other relevant publications . We restricted our search to English- language publications . One review er conducted the search and selected the studies . Systematic review s , defined as review s of published studies with a comprehensive search and systematic selection , were included if they assessed the method ologic quality of the included studies by using 1 or more explicit criteria . We excluded studies if they were meta-analyses of independent patient data only , if their primary goal was to investigate the effectiveness of an intervention or specific diagnostic or screening tests , or if they included studies that were not done on humans . Data Extraction and Synthesis Individual items included in the quality assessment of the systematic review s were recorded as they were reported in the publication ( that is , the information that would be available to readers and future review ers ) . We review ed journal Web sites and contacted the authors of the systematic review s for additional information when authors made such an offer in their original papers . When review s assessed different study design s by using different sets of quality items , we extracted only those items used to assess cohort studies . We constructed a comprehensive list of distinct items that the review s used to assess the quality of their included studies . The full text of each review was screened . All items used by the review authors to assess the quality of studies were extracted into a computerized spreadsheet by 1 review er . Two experienced review ers , a clinical epidemiologist and an epidemiologist , independently synthesized the quality items extracted from the prognosis review s to determine how well the systematic review s assessed potential biases . We did this in 3 steps : 1 ) identified distinct concepts or domains addressed by the quality items ; 2 ) grouped each extracted quality item into the appropriate domain or domains ; and 3 ) identified the domains necessary to assess potential biases in prognosis studies . We then used this information to assess how well the review s ' quality assessment included items from the domains necessary to assess potential biases . After completing each of the first 3 steps , the review ers met to attempt to reach a consensus . The consensus process involved each review er presenting his or her observations and results , followed by discussion and debate . A third review er was available in cases of persistent disagreement or uncertainty . In the first step , all domains addressed by the quality items were identified . The first review er iteratively and progressively defined the domains as items were extracted from the included review s. The second review er defined domains from a r and om list of all extracted quality items . Limited guidance was provided to the review ers so that their assessment s and definitions of domains would be independent . The review ers agreed on a final set of domains that adequately and completely defined all of the extracted items . In the second step , review ers independently grouped each extracted item into the appropriate domains . Review ers considered each extracted item by asking , What is each particular quality item addressing ? or What are the review 's authors getting at with the particular quality assessment item ? . Items were grouped into the domain or domains that best represented the concepts being addressed . For example , the extracted items at least 80 % of the group originally identified was located for follow-up and follow-up was sufficiently complete or does n't jeopardize validity were each independently classified by both review ers as assessing the domain completeness of follow-up adequate , whereas the extracted item quantification and description of all subjects lost to follow-up was classified as assessing the domain completeness of follow-up described . In the third step , we identified the domains necessary to assess potential biases . Each review er considered the ability of the identified domains to adequately address , at least in part , 1 of the following 6 potential biases : 1 ) study participation , 2 ) study attrition , 3 ) prognostic factor measurement , 4 ) confounding measurement and account , 5 ) outcome measurement , and 6 ) analysis . Domains were considered to adequately address part of the framework if information garnered from that domain would inform the assessment of potential bias . For example , both review ers judged that the identified domain study population represents source population or population of interest assessed potential bias in a prognosis study , whereas the domain research question definition did not , although the latter is an important consideration in assessing the inclusion of studies in a systematic review . Finally , on the basis of our previous ratings , we looked at whether each review included items from the domains necessary to assess the 6 potential biases . We calculated the frequency of systematic review s by assessing each potential bias and the number of review s that adequately assessed bias overall . From this systematic synthesis , we developed recommendations for improving quality appraisal in future systematic review s of prognosis studies . We used Microsoft Access and Excel 2002 ( Microsoft Corp. , Redmond , Washington ) for data management and SAS for Windows , version 9.1 ( SAS Institute , Inc. , Cary , North Carolina ) for descriptive statistics . Role of the Funding Sources The funding sources , the Canadian Institutes of Health Research , the Canadian Chiropractic Research Foundation , the Ontario Chiropractic Association , and the Ontario Ministry of Health and Long Term Care , did not have a role in the collection , analysis , or interpretation of the data or in the decision to su bmi t the manuscript for publication . Results We identified 1384 potentially relevant articles . Figure 1 shows a flow chart of studies that were included and excluded . Figure 2 shows the number of review s identified by year of publication . We excluded 131 systematic review s of prognosis studies that did not seem to include any quality assessment of the included studies ; this represented 44 % of prognosis review s. We included 163 review s of prognosis studies in our analysis ( 13 - 175 ) . The most common topics were cancer ( 15 % ) , musculoskeletal disorders and rheumatology ( 13 % ) , cardiovascular ( 10 % ) , neurology ( 10 % ) , and obstetrics ( 10 % ) . Other review s included a wide range of health and health care topics . Sixty-three percent of the review s investigated the association between a specific prognostic factor and a particular outcome ; the remainder investigated multiple prognostic factors or models . The number of primary studies included in each systematic review ranged from 3 to 167 ( median , 18 [ interquartile range , 12 to 31 ] ) . A complete description of the included review s is available from the authors on request . Figure 1 . Flow diagram of inclusion and exclusion criteria of systematic review s. Figure 2 . Number of systematic review s of prognosis studies identified over time . Quality Items One hundred fifty-three review s provided adequate detail to allow extraction of quality items . Eight hundred eighty-two distinct quality items were extracted from the review s. Most review s developed their own set of quality items , with only a few applying criteria from previous review s. Most quality items Background The COSMIN checklist is a tool for evaluating the method ological quality of studies on measurement properties of health-related patient-reported outcomes . The aim of this study is to determine the inter-rater agreement and reliability of each item score of the COSMIN checklist ( n = 114 ) . Methods 75 articles evaluating measurement properties were r and omly selected from the bibliographic data base compiled by the Patient-Reported Outcome Measurement Group , Oxford , UK . Raters were asked to assess the method ological quality of three articles , using the COSMIN checklist . In a one-way design , percentage agreement and intraclass kappa coefficients or quadratic-weighted kappa coefficients were calculated for each item . Results 88 raters participated . Of the 75 selected articles , 26 articles were rated by four to six participants , and 49 by two or three participants . Overall , percentage agreement was appropriate ( 68 % was above 80 % agreement ) , and the kappa coefficients for the COSMIN items were low ( 61 % was below 0.40 , 6 % was above 0.75 ) . Reasons for low inter-rater agreement were need for subjective judgement , and accustom to different st and ards , terminology and definitions . Conclusions Results indicated that raters often choose the same response option , but that it is difficult on item level to distinguish between articles . When using the COSMIN checklist in a systematic review , we recommend getting some training and experience , completing it by two independent raters , and reaching consensus on one final rating . Instructions for using the checklist are improved This paper compares  parent- and  self-report  health-related quality of life ( HRQL ) in children aged 2 - 16 years with brain tumours , one , six and twelve months after diagnosis with matched normal controls .  HRQL was assessed using the PedsQL generic core scales .  37 tumour patients and 42 controls were included in analysis of parent-report  , and 27 patients and 31 controls in self-report HRQL . Parent-report scores were significantly lower in patients than controls for all PedsQL scores at all time points ( max p=0.002 ) . Differences in self-report PedsQL between patients and controls were variable . The relationship between self- and parent-report in patients and controls was inconsistent ; varied over time ; and did not consistently correlate with parental depressive symptoms , suggesting parents and their children do not regard HRQL in a similar way . Prospect i ve , longitudinal assessment of HRQL is important , but should be supplemented with other outcome measures such as health status and behaviour in this population The authors prospect ively assessed anxiety , depression , and behavior in children with st and ard‐risk acute lymphoblastic leukemia ( SR‐ALL ) during the first year of therapy and identified associated risk factors Background A number of r and omized trial studies and longitudinal research es emphasize that despite problems in social adjustment and cognitive damages , children with cancer demonstrate good emotional adjustment . Methods Most of the research findings in this area are obtained using objective tools such as question naires . “ Vitality of children ” , as a drawing tool , was used as a basis to draw a comparison between children with cancer and healthy children in this study . Accordingly , 112 children with cancer ( 5 girls and 57 boys aged 3 to 12 years ) and 123 healthy children ( 77 girls and 46 boys aged 3 to 12 years ) participated in the study . Results Findings showed that the vitality of the two groups differed significantly . Perhaps , children with cancer repress negative emotions and avoid expressing their feelings . MANOVA was used to compare the vitality scores between groups and to explore the impact of different variables . Conclusion Making use of such tools that indirectly examine the emotional experience of children with cancer would be beneficial . Neglecting this issue can cause children with cancer to be deprived of r receiving supportive counselling Hypnosis was compared with nonhypnotic behavioral techniques for efficacy in reducing pain and anxiety in 27 children and adolescents during bone marrow aspiration and in 22 children and adolescents during lumbar puncture . The patients and independent observers each rated ( scale of 1 to 5 ) pain and anxiety during one to three procedures prior to intervention and one to three procedures with intervention . Prior to intervention for both groups , pain during bone marrow aspiration was rated as more severe ( P less than 0.01 ) than pain during lumbar puncture . During bone marrow aspiration pain was reduced to a large extent by hypnosis ( P less than 0.001 ) and to a smaller but significant extent by nonhypnotic techniques ( P less than 0.01 ) , and anxiety was significantly reduced by hypnosis alone ( P less than 0.001 ) . During lumbar puncture only hypnosis significantly reduced pain ( P less than 0.001 ) ; anxiety was reduced to a large degree by hypnosis ( P less than 0.001 ) and to a smaller degree by nonhypnotic techniques ( P less than 0.05 ) . Thus hypnosis was shown to be more effective than nonhypnotic techniques for reducing procedural distress in children and adolescents with cancer Background : Chemotherapy is frequently administered in repetitive cycles . Adolescents with cancer have multiple symptoms related to chemotherapy , but knowledge of symptom trajectories across a cycle is limited . Examining trajectories over a cycle may reveal key periods to manage symptoms . Objectives : The aims of this pilot were to describe the trajectory of symptoms ( pain , sleep , appetite , nausea , fatigue ) and biological and behavioral variables ( anxiety , stress , hematologic function ) across 1 cycle and examine relationships between variables . Methods : Nine adolescents with cancer within 6 months of diagnosis participated . Data were collected by surveys , chart review , and biologic measures on days 1 and 2 of the cycle , 1 week later ( nadir ) , and day 1 of the following cycle . To evaluate the trajectory , a simple r and om-effects repeated- measures analysis was computed . Results : The significant trajectories were fatigue ( P = .003 ) , difficulty sleeping ( P = .032 ) , and nausea ( P = .04 ) . Most of the adolescents reported some anticipatory anxiety about receiving chemotherapy . Significant correlations between symptoms and biobehavioral variables included anticipatory anxiety and nausea ( r = .86 , P = .003 ) , trait anxiety and fatigue ( r = −0.82 , P < .001 ) , and stress and pain ( r = 0.78 , P = .039 ) . Conclusions : Multiple symptoms were experienced across the cycle . Three symptoms displayed significant trajectories indicating that patterns of symptoms may be anticipated . Implication s for Practice : Pilot findings suggest that monitoring symptoms , stress , and anxiety across a cycle is important , not only during chemotherapy administration , but also prior to being admitted for chemotherapy
target: Only three multi-item and two single-item instruments were identified as being reliable and valid among pediatric cancer or HSCT patients and would therefore be appropriate to measure anxiety in this population
background: Purpose The primary objective was to describe anxiety measurement instruments used in children and adolescents with cancer or undergoing hematopoietic stem cell transplantation ( HSCT )  and summarize their content and psychometric properties .