Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. Comparison of registered and published primary outcomes in randomized controlled trials. Funding source, trial outcome and reporting quality: are they related? Results of a pilot study.
Selective reporting of outcomes within published studies based on the nature or direction of their results has been widely suspected, but direct evidence of such bias is currently limited to case reports.
                To study empirically the extent and nature of outcome reporting bias in a cohort of randomized trials.
                Cohort study using protocols and published reports of randomized trials approved by the Scientific-Ethical Committees for Copenhagen and Frederiksberg, Denmark, in 1994-1995. The number and characteristics of reported and unreported trial outcomes were recorded from protocols, journal articles, and a survey of trialists. An outcome was considered incompletely reported if insufficient data were presented in the published articles for meta-analysis. Odds ratios relating the completeness of outcome reporting to statistical significance were calculated for each trial and then pooled to provide an overall estimate of bias. Protocols and published articles were also compared to identify discrepancies in primary outcomes.
                Completeness of reporting of efficacy and harm outcomes and of statistically significant vs nonsignificant outcomes; consistency between primary outcomes defined in the most recent protocols and those defined in published articles.
                One hundred two trials with 122 published journal articles and 3736 outcomes were identified. Overall, 50% of efficacy and 65% of harm outcomes per trial were incompletely reported. Statistically significant outcomes had a higher odds of being fully reported compared with nonsignificant outcomes for both efficacy (pooled odds ratio, 2.4; 95% confidence interval [CI], 1.4-4.0) and harm (pooled odds ratio, 4.7; 95% CI, 1.8-12.0) data. In comparing published articles with protocols, 62% of trials had at least 1 primary outcome that was changed, introduced, or omitted. Eighty-six percent of survey responders (42/49) denied the existence of unreported outcomes despite clear evidence to the contrary.
                The reporting of trial outcomes is not only frequently incomplete but also biased and inconsistent with protocols. Published articles, as well as reviews that incorporate them, may therefore be unreliable and overestimate the benefits of an intervention. To ensure transparency, planned trials should be registered and protocols should be made publicly available prior to trial completion. As of 2005, the International Committee of Medical Journal Editors required investigators to register their trials prior to participant enrollment as a precondition for publishing the trial's findings in member journals.
                To assess the proportion of registered trials with results recently published in journals with high impact factors; to compare the primary outcomes specified in trial registries with those reported in the published articles; and to determine whether primary outcome reporting bias favored significant outcomes.
                MEDLINE via PubMed was searched for reports of randomized controlled trials (RCTs) in 3 medical areas (cardiology, rheumatology, and gastroenterology) indexed in 2008 in the 10 general medical journals and specialty journals with the highest impact factors.
                For each included article, we obtained the trial registration information using a standardized data extraction form.
                Of the 323 included trials, 147 (45.5%) were adequately registered (ie, registered before the end of the trial, with the primary outcome clearly specified). Trial registration was lacking for 89 published reports (27.6%), 45 trials (13.9%) were registered after the completion of the study, 39 (12%) were registered with no or an unclear description of the primary outcome, and 3 (0.9%) were registered after the completion of the study and had an unclear description of the primary outcome. Among articles with trials adequately registered, 31% (46 of 147) showed some evidence of discrepancies between the outcomes registered and the outcomes published. The influence of these discrepancies could be assessed in only half of them and in these statistically significant results were favored in 82.6% (19 of 23).
                Comparison of the primary outcomes of RCTs registered with their subsequent publication indicated that selective outcome reporting is prevalent. There has been increasing concern regarding the potential effects of the commercialization of research.
                In order to examine the relationships between funding source, trial outcome and reporting quality, recent issues of five peer-reviewed, high impact factor, general medical journals were hand-searched to identify a sample of 100 randomized controlled trials (20 trials/journal). Relevant data, including funding source (industry/not-for-profit/mixed/not reported) and statistical significance of primary outcome (favouring new treatment/favouring conventional treatment/neutral/unclear), were abstracted. Quality scores were assigned using the Jadad scale and the adequacy of allocation concealment.
                Sixty-six percent of trials received some industry funding. Trial outcome was not associated with funding source (p=.461). There was a preponderance of favourable statistical conclusions among published trials with 67% reporting results that favored a new treatment whereas 6% favoured the conventional treatment. Quality scores were not associated with funding source or trial outcome.
                It is not known whether the absence of significant associations between funding source, trial outcome and reporting quality reflects a true absence of an association or is an artefact of inadequate statistical power, reliance on voluntary disclosure of funding information, a focus on trials recently published in the top medical journals, or some combination thereof. Continued and expanded monitoring of potential conflicts is recommended, particularly in light of new guidelines for disclosure that have been endorsed by the ICMJE.
Trials published to date lack information on clinically relevant end-points. Existing data is derived from small trials at high risk of bias, which concentrate on physiological measures. Their results are inconclusive. Further large and high quality trials with more appropriate outcomes are warranted.