Publication bias and meta-analyses: a practical example. Effect of including a clinical example on the ability of physical therapists to apply information in a technical research report.
Publication bias is widely appreciated, but considerable time and effort are needed to locate and obtain data from unpublished randomized controlled trials (RCTs), those published in non-English language journals or those reported in the gray literature; for this publication, we will call this collection of trials the "gray+literature." However, excluding such trials from systematic reviews could introduce bias and give rise to misleading conclusions.
                We aimed to explore and quantify the impact of inclusion of gray+ literature on the results of all completed individual patient data (IPD) reviews coordinated by our group (13 meta-analyses). For each IPD review, results were calculated for RCTs fully published in English language journals and RCTs fully published in English language journals and the gray+literature.
                The IPD meta-analyses based only on RCTs that were fully published in English language journals tended to give more favorable results than those that included RCTs from the gray+literature. Although in most cases the addition of gray+data gave less encouraging results, moving the estimated treatment effect toward a null result, the direction of effect was not always predictable.
                We recommend that all systematic reviews should at least attempt to identify trials reported in the gray+literature and, where possible, obtain data from them. The purpose of this study was to determine whether an example in a technical research report would affect application of content to hypothetical patient problems and to future patients.
                Subjects were 69 physical therapists who routinely used isokinetic equipment in the treatment of patients with knee joint pathologies.
                Thirty-five subjects (group 1) read a research report that described mathematical models for predicting preinjury quadriceps femoris and hamstring muscle performance. The report included an example of applying the information. The remaining 34 subjects (group 2) read the same research report with the example omitted. Subjects were asked to select appropriate prediction models and determine preinjury knee torques for two hypothetical patients. Subjects were also contacted 6 to 12 weeks after initial data collection to determine whether they had applied research report results in their treatments of patients.
                Chi-square analyses indicated manuscript type was not related to the selection of correct models, but group 1 subjects computed correct torque values more frequently than did group 2 subjects. Insufficient data were available regarding application of research report content to patients.
                The results indicate application of technically oriented research reports may be enhanced by including examples of applications. (Gross MT, Sekerak DK, Allen DD. Effect of including a clinical example on the ability of physical therapists to apply information in a technical research report.
Although there has been a large amount of research in this area, most studies were excluded due to inherent problems in the nature of their crossover designs. Usually data are not reported before the crossover and the nature of TD and its likely response to treatments makes it imprudent to use this data. The review provides little usable information for service users or providers and more well designed and reported studies are indicated.