Readers' evaluation of effect of peer review and editing on quality of articles in the Nederlands Tijdschrift voor Geneeskunde. Effects of peer review and editing on the readability of articles published in Annals of Internal Medicine. Manuscript quality before and after peer review and editing at Annals of Internal Medicine. A comparison of authors publishing in two groups of U.S. medical journals. Accuracy of references in five biomedical informatics journals. [Effect of statistical review on manuscript quality in Medicina Clínica (Barcelona): a randomized study]. Effect of including a clinical example on the ability of physical therapists to apply information in a technical research report. Comparison of review articles published in peer-reviewed and throwaway journals. Effects of training on quality of peer review: randomised controlled trial. Publication bias and meta-analyses: a practical example. Impact of a medical journal club on house-staff reading habits, knowledge, and critical appraisal skills. A randomized control trial. How does the peer review process influence AANA journal article readability? Effect of attendance at a training session on peer reviewer quality and performance. Does masking author identity improve peer review quality? A randomized controlled trial. PEER Investigators. The effects of blinding on acceptance of research papers by peer review. The Medical Journal of Australia Internet peer-review study. Effect of structured workshop training on subsequent performance of journal peer reviewers. Quotation accuracy: fact or fiction? An intervention to improve the reliability of manuscript reviews for the Journal of the American Academy of Child and Adolescent Psychiatry. Publication bias in presentations to the Annual Scientific Congress. Can the accuracy of abstracts be improved by providing specific instructions? A randomized controlled trial. [Errors in bibliographic references: a retrospective study in Medicina Clinica (1962-1992)]. Effect of written feedback by editors on quality of reviews: two randomized trials. Quality assessment of pharmacoeconomic abstracts of original research articles in selected journals. Commercially funded and United States-based research is more likely to be published; good-quality studies with negative outcomes are not. Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: a randomized controlled trial. Protocol-based computer reminders, the quality of care and the non-perfectability of man. Publication bias: evidence of delayed publication in a cohort study of clinical research projects. The practical effects of errors in reference lists in nursing research journals. Accuracy of data in abstracts of published research articles. Does exchanging comments of Indian and non-Indian reviewers improve the quality of manuscript reviews? Preliminary assessment of the effect of more informative (structured) abstracts on citation retrieval from MEDLINE. Factors associated with failure to publish large randomized trials presented at an oncology meeting. Quotation and reference accuracy of three nursing journals. Citation and quotation accuracy in three anatomy journals. The effects of blinding on the quality of peer review. A randomized trial. Improving drug-therapy decisions through educational outreach. A randomized controlled trial of academically based "detailing". Reporting on covariate adjustment in randomised controlled trials before and after revision of the 2001 CONSORT statement: a literature review. The accuracy of citation and quotation in otolaryngology/head and neck surgery journals. Risk factors for citation errors in peer-reviewed nursing journals. The publication outcome for the papers presented at the 1990 ABA conference. Impact of random assignment on study outcome: an empirical examination. Structured abstracts in MEDLINE, 1989-1991. Effect of blinding and unmasking on the quality of peer review: a randomized trial. Referencing and quotation accuracy in four manual therapy journals. Quality of nonstructured and structured abstracts of original research articles in the British Medical Journal, the Canadian Medical Association Journal and the Journal of the American Medical Association. The accuracy of references in Australian and New Zealand Medical Journals. [The accuracy of references in Turkish journals in orthopedics and traumatology]. Publication rate of abstracts presented at the annual meeting of the American Academy of Otolaryngology-Head and Neck Surgery. A randomized controlled study of reviewer bias against an unconventional therapy. Whatever happened to abstracts from different sections of the association for research in vision and ophthalmology? The impact of an individual tutorial session on MEDLINE use among obstetrics and gynaecology residents in an academic training programme: a randomized trial. How accurate are quotations and references in medical journals? Scientific abstracts at the Society for Adolescent Medicine meetings: a 3-decade comparison. [Errors in literature references in the Nederlands Tijdschrift voor Geneeskunde]. Editorial policy and the reporting of randomized controlled trials: a survey of instructions for authors and assessment of trial reports in Indian medical journals (2004-05). What should be included in meta-analyses? An exploration of methodological issues using the ISPOT meta-analyses. Influence of drug company authorship and sponsorship on drug trial outcomes. Positive-outcome bias and other limitations in the outcome of research abstracts submitted to a scientific meeting. Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. Author, verify your references! Or, the accuracy of references in Israeli medical journals. Searching multiple databases for systematic reviews: added value or diminishing returns? A randomized controlled trial evaluating the impact of knowledge translation and exchange strategies. Discrepancy between results and abstract conclusions in industry- vs nonindustry-funded studies comparing topical prostaglandins. Beyond Medline: reducing bias through extended systematic review search. Absolutely relative: how research results are summarized can affect treatment decisions. Quality of structured abstracts of original research articles in the British Medical Journal, the Canadian Medical Association Journal and the Journal of the American Medical Association: a 10-year follow-up study. Effect of open peer review on quality of reviews and on reviewers' recommendations: a randomised trial. An observational study of orthopaedic abstracts and subsequent full-text publications. Quality of randomized controlled trials reporting in the primary treatment of brain tumors. Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough. The effects of two continuing medical education programs on communication skills of practicing primary care physicians. [Errors in bibliographic references of Nefrología from 1981 to 1995. A quality control].
Academic biomedical journals use peer review and editing to help to select and improve the quality of articles. We have investigated whether articles accepted by the Nederlands Tijdschrift voor Geneeskunde, the Dutch Journal of Medicine, were improved after peer review and editing (post-acceptance scientific and copy editing).
                400 readers of the journal (100 each of medical students, recent medical graduates, general practitioners, and specialists) were invited to participate in a questionnaire survey. The first 25 from each group who agreed to participate were included. We posted a pack containing a set of identically appearing typescripts (ie, blinding) of the submitted, accepted, and published versions of 50 articles that had been published in Ned Tijdschr Geneeskd. Each evaluator received two of the sets of versions, and each set was evaluated by one person from each group. The package also included two questionnaires: the first was used to compare the submitted with the accepted version (25 questions), the second compared the accepted with the published version (17 questions). The questions were answered on five-point scales, and were about the quality of the articles or were general/overall scores. We analysed the data as scores of 3-5 (ie, improvement) versus 1-2.
                After peer review, the quality in 14 of 23 questions (61%) was significantly improved (p = 0.03 or smaller). In particular, the overall score and general medical value were significantly improved (p = 0.00001 for each). Editing led to significant improvement in 11 of 16 questions (69%, p = 0.017 or smaller), and especially in style and readability (p = 0.001 and p = 0.004). Generally, we found no differences between the scores of the four categories of evaluators. 72% of the evaluators correctly identified which version was which.
                Evaluations by readers of the Ned Tijdschr Geneeskd indicated significant improvement of published articles after both peer review and editing. We think that peer review and editing are worthwhile tasks. We also think that possible biases would have had a negligible effect on our results (including the fact that we selected the first 25 evaluators who responded, that some evaluators may have read the published version, and that one questionnaire may have looked more scientific than the other, more editorial one). To measure the effect of the peer review and editorial processes on the readability of original articles.
                Comparison of manuscripts before and after the peer review and editorial processes.
                Annals of Internal Medicine between March 1 and November 30, 1992. MANUSCRIPTS: One hundred one consecutive manuscripts reporting original research.
                Assessment of readability by means of two previously validated indexes: the Gunning fog index (units of readability in the fog index roughly correlate to years of education) and the Flesch reading ease score. Each manuscript was analyzed for readability and length on receipt and after it had passed through the peer review and editorial processes. Text and abstracts were analyzed similarly but separately. Mean readability scores were compared by two-tailed t tests for paired observations.
                Mean (+/- SD) initial readability scores of manuscripts and abstracts by the Gunning fog index were 17.16 +/- 1.55 and 16.65 +/- 2.80, respectively. At publication, scores were 16.85 +/- 1.42 and 15.64 +/- 2.42 (P = .0005 and P < .0001 for before-after differences, respectively). By comparison, studies of other print media showed scores of about 11 for the New York Times editorial page and about 18 for a typical legal contract. Similar changes were found for the Flesch scores. The median length of the manuscripts increased by 2.6% and that of the abstracts by 4.2% during the processes.
                The peer review and editorial processes slightly improved the readability of original articles and their abstracts, but both remained difficult to read at publication. Better readability scores may improve readership. To evaluate the effects of peer review and editing on manuscript quality.
                Editorial offices of Annals of Internal Medicine.
                Masked before-after study. MANUSCRIPTS: 111 consecutive original research manuscripts accepted for publication at Annals between March 1992 and March 1993.
                We used a manuscript quality assessment tool of 34 items to evaluate the quality of the research report, not the quality of the research itself. Each item was scored on a 1 to 5 scale. Forty-four expert assessors unaware of the design or aims of the study evaluated the manuscripts, with different persons evaluating the two versions of each manuscript (before and after the editorial process).
                33 of the 34 items changed in the direction of improvement, with the largest improvements seen in the discussion of study limitations, generalizations, use of confidence intervals, and the tone of conclusions. Overall, the percentage of items scored three or more increased by an absolute 7.3% (95% CI, 3.3% to 11.3%) from a baseline of 75%. The average item score improved by 0.23 points (CI, 0.07 to 0.39) from a baseline mean of 3.5. Manuscripts rated in the bottom 50% showed two- to threefold larger improvements than those in the top 50%, after correction for regression to the mean.
                Peer review and editing improve the quality of medical research reporting, particularly in those areas that readers rely on most heavily to decide on the importance and generalizability of the findings. This study compared the editorial peer review experiences of authors who published in two groups of indexed U.S. medical journals. The study tested the hypothesis that after one journal rejects a manuscript an author selects a less well-known journal for submission. Group One journals were defined as those indexed in 1992 MEDLINE that satisfied several additional qualitative measures; Group Two journals were indexed in the 1992 MEDLINE only. Surveys were sent to the first authors of 616 randomly selected articles, and 479 surveys were returned, for a response rate of 78.1%. A total of 20.8% of Group One and 15.7% of Group Two articles previously had been rejected. Group One authors were more likely to select a journal for its prestige and article quality, while Group Two authors were more likely to have been invited to submit the manuscript. More than 60% of both groups felt the peer review had offered constructive suggestions, but that it had changed article conclusions less than 3% of the time. Both groups thought the review process only marginally improved content, organization, or statistical analysis, or clarified conclusions. Between 3% and 15% of all authors received considerable conflicting advice from different reviewers. Authors from both groups differed as to their reasons for journal selection, their connections with the publishing journal, and patterns of resubmission after rejection. To determine the rate and type of errors in biomedical informatics journal article references.
                References in articles from the first 2004 issues of five biomedical informatics journals, Journal of the American Medical Informatics Association, Journal of Biomedical Informatics, International Journal of Medical Informatics, Methods of Information in Medicine, and Artificial Intelligence in Medicine were compared with MEDLINE for journal, authors, title, year, volume, and page number accuracy. If discrepancies were identified, the reference was compared with the original publication. Two reviewers independently evaluated each reference.
                The five journal issues contained 37 articles. Among the 656 eligible references, 225 (34.3%) included at least one error. Among the 225 references, 311 errors were identified. One or more errors were found in the bibliography of 31 (84%) of the 37 articles. The reference error rates by journal ranged from 22.1% to 40.7%. Most errors (39.0%) occurred in the author element, followed by the journal (31.2%), title (17.7%), page (7.4%), year (3.5%), and volume (1.3%) information.
                The study identified a considerable error rate in the references of five biomedical informatics journals. Authors are responsible for the accuracy of references and should more carefully check them, possibly using informatics-based assistance. The statistical review of biomedical articles should result in an improved quality. The objective of this study was to compare the effects of clinical review and joint clinical and statistical review on manuscript quality, in articles submitted to Medicina Clínica (Barcelona), a Spanish weekly journal of internal medicine.
                Original papers arriving between May 2000 and February 2001 were randomized either to a clinical review group or a clinical and statistical review group. Two evaluators, blinded to the paper's group, assessed the quality improvement in both groups, from submission to publication using a modified version ot the Goodman et al. scale. The protocol required that final versions arrived before the end of May 2001.
                Final sample size was 43 manuscripts, evaluated before and after peer review. On the intention to treat analysis, the estimated effect of statistical review was 1.35 (95% CI: -0.45 to 3.16) positive, but not statistically significant. The analysis of the reviewers' comments revealed some protocol deviations. Taking into account the spontaneous inclusion of statistical experts in the clinical group, the estimated effect was statistically significant, with a confidence interval of 0.3 to 3.7.
                The inclusion of a statistical expert in the peer review process improves manuscript quality, although in the intention to treat analysis the improvement was not statistically significant. The purpose of this study was to determine whether an example in a technical research report would affect application of content to hypothetical patient problems and to future patients.
                Subjects were 69 physical therapists who routinely used isokinetic equipment in the treatment of patients with knee joint pathologies.
                Thirty-five subjects (group 1) read a research report that described mathematical models for predicting preinjury quadriceps femoris and hamstring muscle performance. The report included an example of applying the information. The remaining 34 subjects (group 2) read the same research report with the example omitted. Subjects were asked to select appropriate prediction models and determine preinjury knee torques for two hypothetical patients. Subjects were also contacted 6 to 12 weeks after initial data collection to determine whether they had applied research report results in their treatments of patients.
                Chi-square analyses indicated manuscript type was not related to the selection of correct models, but group 1 subjects computed correct torque values more frequently than did group 2 subjects. Insufficient data were available regarding application of research report content to patients.
                The results indicate application of technically oriented research reports may be enhanced by including examples of applications. (Gross MT, Sekerak DK, Allen DD. Effect of including a clinical example on the ability of physical therapists to apply information in a technical research report. To compare the quality, presentation, readability, and clinical relevance of review articles published in peer-reviewed and "throwaway" journals.
                We reviewed articles that focused on the diagnosis or treatment of a medical condition published between January 1 and December 31, 1998, in the 5 leading peer-reviewed general medical journals and high-circulation throwaway journals. Reviewers independently assessed the methodologic and reporting quality, and evaluated each article's presentation and readability. Clinical relevance was evaluated independently by 6 physicians.
                Of the 394 articles in our sample, 16 (4.1%) were peer-reviewed systematic reviews, 135 (34.3%) were peer-reviewed nonsystematic reviews, and 243 (61.7%) were nonsystematic reviews published in throwaway journals. The mean (SD) quality scores were highest for peer-reviewed articles (0.94 [0.09] for systematic reviews and 0.30 [0.19] for nonsystematic reviews) compared with throwaway journal articles (0.23 [0.03], F(2,391) = 280.8, P<.001). Throwaway journal articles used more tables (P =.02), figures (P =.01), photographs (P<.001), color (P<.001), and larger font sizes (P<.001) compared with peer-reviewed articles. Readability scores were more often in the college or higher range for peer-reviewed journals compared with the throwaway journal articles (104 [77.0%] vs 156 [64.2%]; P =.01). Peer-reviewed article titles were judged less relevant to clinical practice than throwaway journal article titles (P<.001).
                Although lower in methodologic and reporting quality, review articles published in throwaway journals have characteristics that appeal to physician readers. To determine the effects of training on the quality of peer review.
                Single blind randomised controlled trial with two intervention groups receiving different types of training plus a control group.
                Reviewers at a general medical journal. Interventions Attendance at a training workshop or reception of a self taught training package focusing on what editors want from reviewers and how to critically appraise randomised controlled trials.
                Quality of reviews of three manuscripts sent to reviewers at four to six monthly intervals, evaluated using the validated review quality instrument; number of deliberate major errors identified; time taken to review the manuscripts; proportion recommending rejection of the manuscripts.
                Reviewers in the self taught group scored higher in review quality after training than did the control group (score 2.85 v 2.56; difference 0.29, 95% confidence interval 0.14 to 0.44; P = 0.001), but the difference was not of editorial significance and was not maintained in the long term. Both intervention groups identified significantly more major errors after training than did the control group (3.14 and 2.96 v 2.13; P < 0.001), and this remained significant after the reviewers' performance at baseline assessment was taken into account. The evidence for benefit of training was no longer apparent on further testing six months after the interventions. Training had no impact on the time taken to review the papers but was associated with an increased likelihood of recommending rejection (92% and 84% v 76%; P = 0.002).
                Short training packages have only a slight impact on the quality of peer review. The value of longer interventions needs to be assessed. Publication bias is widely appreciated, but considerable time and effort are needed to locate and obtain data from unpublished randomized controlled trials (RCTs), those published in non-English language journals or those reported in the gray literature; for this publication, we will call this collection of trials the "gray+literature." However, excluding such trials from systematic reviews could introduce bias and give rise to misleading conclusions.
                We aimed to explore and quantify the impact of inclusion of gray+ literature on the results of all completed individual patient data (IPD) reviews coordinated by our group (13 meta-analyses). For each IPD review, results were calculated for RCTs fully published in English language journals and RCTs fully published in English language journals and the gray+literature.
                The IPD meta-analyses based only on RCTs that were fully published in English language journals tended to give more favorable results than those that included RCTs from the gray+literature. Although in most cases the addition of gray+data gave less encouraging results, moving the estimated treatment effect toward a null result, the direction of effect was not always predictable.
                We recommend that all systematic reviews should at least attempt to identify trials reported in the gray+literature and, where possible, obtain data from them. The journal club is an established teaching modality in many house-staff training programs. To determine if a journal club improves house-staff reading habits, knowledge of epidemiology and biostatistics, and critical appraisal skills, we randomized 44 medical interns to receive either a journal club or a control seminar series. A test instrument developed by the Delphi method was administered before and after the interventions (mean, five journal club sessions). By self-report, 86% of the house staff in the journal club group improved their reading habits vs 0% in the control group. Knowledge scores increased more in the journal club group than in the control group, and a trend was found toward more knowledge gained as more sessions were attended. Ability to appraise critically a test article increased slightly in each group, but there was no significant difference between the groups. We conclude that a journal club is a powerful motivator of critical house-staff reading behavior and can help teach epidemiology and biostatistics to physicians-in-training. This study examined the readability of the AANA Journal, quantifying the effect of peer review on case and research reports published from 1992 to 1994. Gunning and Flesch index-based computer analysis, as well as human comparative analysis, was undertaken. Computer and human assessment of readability revealed improvement as papers evolved from submitted to published versions; however, at publication the manuscripts remained in the "difficult" readability range. Although this study provides evidence that peer review improves readability, it may be that, due to a professed need for scientific purity and an imposed sense of scholarship, nursing and other biomedical journals may overemphasize a style and approach that paradoxically make transfer of information unreasonably difficult. To determine whether attendance at a voluntary training workshop improves quality ratings of medical journal peer reviewers.
                Peer reviewers for Annals of Emergency Medicine who completed two or more reviews during the 20 months before or the 20 months after October 1995 were eligible. Reviews were routinely rated by editors on a subjective 5-point quality scale. Comparisons were made between reviewers who chose to attend a 4-hour workshop on peer review sponsored by the journal in 1995 (attendees) and 2 groups of reviewers who did not attend: controls matched for review quality and number of reviews completed before the workshop, and unmatched controls. Guest reviewers were excluded.
                A total of 298 reviewers completed 1906 reviews before the workshop and 2,194 after the workshop; 2,117 of these reviews were rated by editors. Forty-five attendees participated in the workshop, 39 of whom had sufficient ratings for analysis. Matched controls were almost identical in performance to attendees, but unmatched controls had performed fewer reviews and had lower average ratings before the workshop. There was no significant change in any performance measurement after the workshop, including average quality rating, percent change in quality rating, odds ratio for recommending acceptance, and odds ratio for congruence with editor's decision.
                In a self-selected group of experienced reviewers who attended a 4-hour workshop on peer review, no effect could be identified in subsequent performance as measured by editors' quality ratings or reviewer performance statistics. All authors may not be equal in the eyes of reviewers. Specifically, well-known authors may receive less objective (poorer quality) reviews. One study at a single journal found a small improvement in review quality when reviewers were masked to author identity.
                To determine whether masking reviewers to author identity is generally associated with higher quality of review at biomedical journals, and to determine the success of routine masking techniques.
                A randomized controlled trial performed on external reviews of manuscripts submitted to Annals of Emergency Medicine, Annals of Internal Medicine, JAMA, Obstetrics & Gynecology, and Ophthalmology.
                Two peers reviewed each manuscript. In one study arm, both peer reviewers received the manuscript according to usual masking practice. In the other arm, one reviewer was randomized to receive a manuscript with author identity masked, and the other reviewer received an unmasked manuscript.
                Review quality on a 5-point Likert scale as judged by manuscript author and editor. A difference of 0.5 or greater was considered important.
                A total of 118 manuscripts were randomized, 26 to usual practice and 92 to intervention. In the intervention arm, editor quality assessment was complete for 77 (84%) of 92 manuscripts. Author quality assessment was complete on 40 (54%) of 74 manuscripts. Authors and editors perceived no significant difference in quality between masked (mean difference, 0.1; 95% confidence interval [CI], -0.2 to 0.4) and unmasked (mean difference, -0.1; 95% CI, -0.5 to 0.4) reviews. We also found no difference in the degree to which the review influenced the editorial decision (mean difference, -0.1; 95% CI,-0.3 to 0.3). Masking was often unsuccessful (overall, 68% successfully masked; 95% CI, 58%-77%), although 1 journal had significantly better masking success than others (90% successfully masked; 95% CI, 73%-98%). Manuscripts by generally known authors were less likely to be successfully masked (odds ratio, 0.3; 95% CI, 0.1-0.8). When analysis was restricted to manuscripts that were successfully masked, review quality as assessed by editors and authors still did not differ.
                Masking reviewers to author identity as commonly practiced does not improve quality of reviews. Since manuscripts of well-known authors are more difficult to mask, and those manuscripts may be more likely to benefit from masking, the inability to mask reviewers to the identity of well-known authors may have contributed to the lack of effect. To study whether reviewers aware of author identity are biased in favor of authors with more previous publications.
                Randomized controlled trial.
                Editorial office of the Journal of Developmental and Behavioral Pediatrics.
                Two "blinded" and two "nonblinded" reviewers assigned to 57 consecutive manuscripts submitted between September 1991 and March 1992.
                Spearman rank correlation coefficients were used to compare the sum of rating scores of 1 to 5 (1, accept; 5, reject) given by the two blinded reviewers, the two nonblinded reviewers, and the editors to the number of articles published previously by the first and senior authors (as determined from requested curricula vitae). Blinded reviewers were sent a questionnaire asking whether they could determine the identity of the authors, how they knew, and whether they thought binding changed the quality or difficulty of their review.
                The Wilcoxon Sign Rank Test disclosed no differences between blinded and nonblinded scores. The number of previous articles by the senior author was significantly correlated (P < .01) with blinded scores (r = -.45) and editors' decisions (r = -.45), but not with nonblinded scores; the number of articles by the first author was correlated (P < .05) with editors' decisions (r = -.35) but not with blinded or nonblinded scores. Fifty (46%) of 108 blinded reviewers correctly guessed the identity of the authors, mostly from self-references and knowledge of the work; 86% believed blinding did not change the quality of their review, and 73% believed it did not change the difficulty of performing a review.
                Blinded reviewers and editors in this study, but not nonblinded reviewers, gave better scores to authors with more previous articles. These results suggest that blinded reviewers may provide more unbiased reviews and that nonblinded reviewers may be affected by various types of bias. Peer review of medical papers is a confidential consultancy between the reviewer and the journal editor, and has been criticised for its potential bias and inadequacy. We explored the potential of the internet for open peer review to see whether this approach improved the quality and outcome of peer review.
                Research and review articles that had been accepted for publication in The Medical Journal of Australia (MJA) were published together with the reviewers' reports on the worldwide web, with the consent of authors and referees. Selected readers' e-mailed comments were electronically published as additional commentary; authors could reply or revise their paper in response to readers' comments. Articles were edited and published in print after this open review.
                60 (81%) of 74 authors agreed to take part in the study, together with 150 (92%) of 162 reviewers. There was no significant difference in the performance of commissioned reviewers before and during the study. Four articles were not included because of insufficient time before print publication. Of the remaining 56 papers, 28 received 52 comments from 42 readers (2% of readers submitted comments). Most readers' comments were short and specific, and seven articles were changed by the authors in response.
                Open peer review is acceptable to most authors and reviewers. Postpublication review by readers on the internet is no substitute for commissioned prepublication review, but can provide editors with valuable input from individuals who would not otherwise be consulted. Readers also gain insight into the processes of peer review and publication. Study objective: We sought to determine whether peer reviewers who attend a formal interactive training session produce better reviews.
                Peer reviewers were invited to attend a formal, 4-hour, highly interactive workshop on peer review. Attendees received a sample manuscript to read and review in writing in advance. The workshop included presentations on analyzing a study and the journal's expectations for a quality review, discussion of the sample manuscript's flaws and how to address them in a review, discussion of the reviews written by the attendees, and discussion of real reviews of other manuscripts illustrating key points. The performance of attendees on the basis of standard editor quality ratings (1 to 5) was assessed for the 2 years after workshop attendance. Control reviewers matched for previous review quality and volume were selected from nonattendees of the workshop. In study 1, all average reviewers received a standard written invitation. In study 2, 75 randomly selected average reviewers were personally and actively recruited with intensive follow-up by means of e-mail and telephone calls in an effort to reduce self-selection bias.
                In study 1, 25 reviewers volunteered for the course, were eligible for study, attended, and were compared with 25 matched control reviewers. Of attendees filling out evaluations, 19% thought it somewhat and 81% thought it very helpful. All thought it would improve their subsequent reviews, and 85% thought it would improve their review ratings. The mean change in rating after the workshop was 0.11 (95% confidence interval [CI] -0.25 to 0.48) for control reviewers and 0.10 (95% CI -0.20 to 0.39) for attendees. In study 2, of 75 reviewers intensively recruited, only 12 (41%) of those who said they would attend did. All of the participants thought the workshop would improve their performance and ratings. Test scores at the end of the workshop improved in 73% of participants compared with scores on pretests. The control reviewers' average rating changed by -0.10 (95% CI -0.49 to 0.29) versus 0.06 (95% CI -0.34 to 0.23) for attendees.
                Among invited peer reviewers, voluntary attendance at a highly structured and interactive workshop was low and did not improve the quality of subsequent reviews, contrary to the predictions of attendees. Efforts to aggressively recruit average reviewers to a second workshop were time consuming, had low success rates, and showed a similar lack of effect on ratings, despite improvement in scores on a test instrument. Workshop teaching formats, although traditional, are of unproven efficacy. Results from original studies are not always correctly represented in subsequent medical publications. In this study, randomly selected quotations from Israeli medical journals were examined. Less than 80% of the examined statements were in complete agreement with the original source. In about 7% the cited reference failed to substantiate the previous author's statement. Similar results were found over a range of American and European general and speciality journals. It is concluded that errors in quotation are a widespread problem in the biomedical literature. Editors and peer reviewers can improve quotation accuracy by stressing this issue in decisions regarding publication. Nevertheless, it seems that the most efficient solution would be for authors to increase their vigilance in accurately quoting results and statements from previous publications. The effects of methods used to improve the interrater reliability of reviewers' ratings of manuscripts submitted to the Journal of the American Academy of Child and Adolescent Psychiatry were studied.
                Reviewers' ratings of consecutive manuscripts submitted over approximately 1 year were first analyzed; 296 pairs of ratings were studied. Intraclass correlations and confidence intervals for the correlations were computed for the two main ratings by which reviewers quantified the quality of the article: a 1-10 overall quality rating and a recommendation for acceptance or rejection with four possibilities along that continuum. Modifications were then introduced, including a multi-item rating scale and two training manuals to accompany it. Over the next year, 272 more articles were rated, and reliabilities were computed for the new scale and for the scales previously used.
                The intraclass correlation of the most reliable rating before the intervention was 0.27; the reliability of the new rating procedure was 0.43. The difference between these two was significant. The reliability for the new rating scale was in the fair to good range, and it became even better when the ratings of the two reviewers were averaged and the reliability stepped up by the Spearman-Brown formula. The new rating scale had excellent internal consistency and correlated highly with other quality ratings.
                The data confirm that the reliability of ratings of scientific articles may be improved by increasing the number of rating scale points, eliciting ratings of separate, concrete items rather than a global judgment, using training manuals, and averaging the scores of multiple reviewers. Free papers presented to the Annual Scientific Congress (ASC) of the Royal Australasian College of Surgeons (RACS) were reviewed for the years 1994, 1995 and 1996. Reports were examined for evidence of publication bias.
                Suitable free papers were identified from the proceedings of the meetings and authors were contacted to obtain information about the research reported and any publications resulting from it.
                Responses were obtained from 302 of 576 presentations considered suitable. A total of 55% of responding authors reported publication of their paper. Basic science papers were most likely to be published. There was a significant bias in favour of publication of positive results (98 of 139 positive vs 76 of 159 inconclusive or negative reports; P < 0.01). Retrospective data were as likely to be published as prospective (51% and 57%, respectively). Reports describing studies of high-level evidence were more likely to be published in journals with a high impact factor.
                The ASC is a comprehensive meeting that attracts a wide range of free papers from most sections of the RACS. There appears to be no evidence of bias in selection of papers for inclusion in the meeting but there is bias in the subsequent publication, which favours positive reports. The most-read section of a research article is the abstract, and therefore it is especially important that the abstract be accurate.
                To test the hypothesis that providing authors with specific instructions about abstract accuracy will result in improved accuracy.
                Randomized controlled trial of an educational intervention specifying 3 types of common defects in abstracts of articles that had been reviewed and were being returned to the authors with an invitation to revise.
                Proportion of abstracts containing 1 or more of the following defects: inconsistency in data between abstract and body of manuscript (text, tables, and figures), data or other information given in abstract but not in body, and/or conclusions not justified by information in the abstract.
                Of 250 manuscripts randomized, 13 were never revised and 34 were lost to follow-up, leaving a final comparison between 89 in the intervention group and 114 in the control group. Abstracts were defective in 25 (28%) and 30 (26%) cases, respectively (P=.78). Among 55 defective abstracts, 28 (51%) had inconsistencies, 16 (29%) contained data not present in the body, 8 (15%) had both types of defects, and 3 (5%) contained unjustified conclusions.
                Defects in abstracts, particularly inconsistencies between abstract and body and the presentation of data in abstract but not in body, occur frequently. Specific instructions to authors who are revising their manuscripts are ineffective in lowering this rate. Journals should include in their editing processes specific and detailed attention to abstracts. Misspellings of transcription or the copy of references of articles published in the literature without verifying the information with the original document may be the cause of incorrect citations. The purpose of this study was to determine the accuracy of citations to journal articles in 1962 to 1992 volumes of Medicina Clínica.
                A method of systematic sampling was used to choose five references for each of the 12 original articles per year selected at 5-year intervals (1962, 1967, 1972, 1977, 1982, 1987, 1992). In the final sample of 420 references, errors in the six standard bibliographic elements--authors' names, article title, journal title, year, volume number, and page numbers--were analyzed. Major errors were defined when inaccuracies prevented identification of the cited article.
                Of the total 368 references verified, 132 (35.86%) were error free and the remaining 236 (64, 13%) were inaccurate. Multiple errors were found in 45% of cases. The mean number of errors per references was 1.13. Twelve references were incorrect, 221 errors were found in the authors' name, 131 in the title of the article, 14 in the name of the journal, 5 in the year of publication, 12 in the number of the volume, and 22 in the initial or final pages of the article. Major errors were detected in 94 (22.54%) cases and minor errors in 323 (77.45%). There were no statistically significant differences in the variation of the overall percentage of errors throughout the study period.
                A high rate of citation errors has been also documented in a Spanish medical journal. The importance of reference verification should be emphasized and authors should exercise more care in preparing bibliographies of their articles. Better peer review is needed, but proven methods to improve quality are unknown. Our objective was to determine whether written feedback to reviewers improves subsequent reviews.
                Eligible reviewers were randomized to intervention or control (receiving other reviewers' unscored reviews and the editor's decision letter). Study 1 (September 1998-September 2000) included reviewers with a median quality score of 3 or lower; study 2 (April 2000-January 2002), reviewers with median score of 4 or lower. Study 1 was designed with a power of 0.80 to detect a difference in score of 1; study 2, with a power of 0.80 to detect a difference of 0.5. All reviewers were at a peer-reviewed journal (Annals of Emergency Medicine). The main outcome measure was the editor's routine quality rating (1-5) of all reviews (blinded to study enrollment).
                For study 1, 51 reviewers were eligible and randomized and 35 had sufficient data (182 reviews) for analysis. The mean individual reviewer rating change was 0.16 (95% confidence interval [CI], -0.26 to 0.58) for control and -0.13 (-0.49 to 0.23) for intervention. For study 2, 127 reviewers were eligible and randomized, and 95 had sufficient data (324 reviews). Controls had a mean individual rating change of 0.12 (95% CI, -0.20 to 0.26) and intervention reviewers, 0.06 (-0.19 to 0.31).
                In study 1, minimal feedback from editors on review quality had no effect on subsequent performance of poor-quality reviewers, and the trend was toward a negative effect. In study 2, feedback to average reviewers was more extensive and supportive but produced no improvement in reviewer performance. Simple written feedback to reviewers seems to be an ineffective educational tool. To assess and compare the quality of pharmacoeconomic abstracts of cost-minimization analyses, cost-effectiveness analyses, cost-utility analyses, and cost-benefit analyses of original research articles in selected medical, pharmacy, and health economics journals.
                MEDLINE was used to identify articles in selected medical, pharmacy, and health economics journals using the MeSH word "economic" and text words "cost" and "pharmacoeconomic"; the journal PharmacoEconomics was searched manually. All retrieved abstracts were evaluated. Original, comparative (at least one drug comparator) research articles (1990-1994) reporting both costs and clinical outcomes were included in the quality analysis. Abstract quality was assessed as a percentage by using a checklist with 29 objective criteria. Group consensus produced interrater reliability greater than 0.8.
                One thousand two published abstracts labeled with the above key words were identified. Of these, 951 were excluded from quality assessment because they were not original research (18%), were not pharmacoeconomic research (47%), lacked a drug comparator (35%), or did not report a clinical outcome (0.5%). Thus, the quality of 51 (5% of the total) remaining abstracts was assessed. Overall scores were 56% in 1990 and 58% in 1994 (p = 0.094). Medical articles scored highest (61.5%; n = 25), pharmacy articles were next (54.3%; n = 5), and health economics articles were lowest (53.4%; n = 21) (p = 0.091); structured abstracts scored significantly higher (62.5%; n = 20) than unstructured (53.3%; n = 31) (p = 0.003).
                Abstract quality was generally poor, with no significant change in quality over time. Medical journals scored highest, probably because they use structured abstracts. Guidelines for structured pharmacoeconomic abstracts may assist in improving quality. Prior studies implying associations between receipt of commercial funding and positive (significant and/or pro-industry) research outcomes have analyzed only published papers, which is an insufficiently robust approach for assessing publication bias. In this study, we tested the following hypotheses regarding orthopaedic manuscripts submitted for review: (1) nonscientific variables, including receipt of commercial funding, affect the likelihood that a peer-reviewed submission will conclude with a report of a positive study outcome, and (2) positive outcomes and other, nonscientific variables are associated with acceptance for publication.
                All manuscripts about hip or knee arthroplasty that were submitted to The Journal of Bone and Joint Surgery, American Volume, over seventeen months were evaluated to determine the study design, quality, and outcome. Analyses were carried out to identify associations between scientific factors (sample size, study quality, and level of evidence) and study outcome as well as between non-scientific factors (funding source and country of origin) and study outcome. Analyses were also performed to determine whether outcome, scientific factors, or nonscientific variables were associated with acceptance for publication.
                Two hundred and nine manuscripts were reviewed. Commercial funding was not found to be associated with a positive study outcome (p = 0.668). Studies with a positive outcome were no more likely to be published than were those with a negative outcome (p = 0.410). Studies with a negative outcome were of higher quality (p = 0.003) and included larger sample sizes (p = 0.05). Commercially funded (p = 0.027) and United States-based (p = 0.020) studies were more likely to be published, even though those studies were not associated with higher quality, larger sample sizes, or lower levels of evidence (p = 0.24 to 0.79).
                Commercially funded studies submitted for review were not more likely to conclude with a positive outcome than were nonfunded studies, and studies with a positive outcome were no more likely to be published than were studies with a negative outcome. These findings contradict those of most previous analyses of published (rather than submitted) research. Commercial funding and the country of origin predict publication following peer review beyond what would be expected on the basis of study quality. Studies with a negative outcome, although seemingly superior in quality, fared no better than studies with a positive outcome in the peer-review process; this may result in inflation of apparent treatment effects when the published literature is subjected to meta-analysis. Anxiety about bias, lack of accountability, and poor quality of peer review has led to questions about the imbalance in anonymity between reviewers and authors.
                To evaluate the effect on the quality of peer review of blinding reviewers to the authors' identities and requiring reviewers to sign their reports.
                Randomized controlled trial.
                A general medical journal.
                A total of 420 reviewers from the journal's database.
                We modified a paper accepted for publication introducing 8 areas of weakness. Reviewers were randomly allocated to 5 groups. Groups 1 and 2 received manuscripts from which the authors' names and affiliations had been removed, while groups 3 and 4 were aware of the authors' identities. Groups 1 and 3 were asked to sign their reports, while groups 2 and 4 were asked to return their reports unsigned. The fifth group was sent the paper in the usual manner of the journal, with authors' identities revealed and a request to comment anonymously. Group 5 differed from group 4 only in that its members were unaware that they were taking part in a study.
                The number of weaknesses in the paper that were commented on by the reviewers.
                Reports were received from 221 reviewers (53%). The mean number of weaknesses commented on was 2 (1.7, 2.1, 1.8, and 1.9 for groups 1, 2, 3, and 4 and 5 combined, respectively). There were no statistically significant differences between groups in their performance. Reviewers who were blinded to authors' dentities were less likely to recommend rejection than those who were aware of the authors' identities (odds ratio, 0.5; 95% confidence interval, 0.3-1.0).
                Neither blinding reviewers to the authors and origin of the paper nor requiring them to sign their reports had any effect on rate of detection of errors. Such measures are unlikely to improve the quality of peer review reports. To determine whether clinical errors can be reduced by prospective computer suggestions about the management of simple clinical events, I studied the responses of nine physicians to computer suggestions generated by 390 protocols in a controlled crossover design. These protocols dealt primarily with conditions managed (e.g., elevated blood pressure) or caused (e.g., liver toxicity) by drugs. Physicians responded to 51 per cent of 327 events when given, and 22 per cent of 385 events when not given computer suggestions. Neither level of postgraduate training (first-year postgraduate or third-year post-graduate) nor the order in which physicians served as study and control subjects had statistically significant overall effect on the results. It appears that the prospective reminders do reduce errors, and that many of these errors are probably due to man's limitations as a data processor rather than to correctable human deficiencies. To determine the extent to which publication is influenced by study outcome.
                A cohort of studies submitted to a hospital ethics committee over 10 years were examined retrospectively by reviewing the protocols and by questionnaire. The primary method of analysis was Cox's proportional hazards model.
                University hospital, Sydney, Australia.
                748 eligible studies submitted to Royal Prince Alfred Hospital Ethics Committee between 1979 and 1988.
                Time to publication.
                Response to the questionnaire was received for 520 (70%) of the eligible studies. Of the 218 studies analysed with tests of significance, those with positive results (P < 0.05) were much more likely to be published than those with negative results (P > or = 0.10) (hazard ratio 2.32 (95% confidence interval 1.47 to 3.66), P = 0.0003), with a significantly shorter time to publication (median 4.8 v 8.0 years). This finding was even stronger for the group of 130 clinical trials (hazard ratio 3.13 (1.76 to 5.58). P = 0.0001), with median times to publication of 4.7 and 8.0 years respectively. These results were not materially changed after adjusting for other significant predictors of publication. Studies with indefinite conclusions (0.05 < or = P < 0.10) tended to have an even lower publication rate and longer time to publication than studies with negative results (hazard ratio 0.39 (0.13 to 1.12), P = 0.08). For the 103 studies in which outcome was rated qualitatively, there was no clear cut evidence of publication bias, although the number of studies in this group was not large.
                This study confirms the evidence of publication bias found in other studies and identifies delay in publication as an additional important factor. The study results support the need for prospective registration of trials to avoid publication bias and also support restricting the selection of trials to those started before a common date in undertaking systematic reviews. Previous studies have noted the prevalence of errors in journal reference lists, including nursing journals, but an in-depth study of nursing research journals has not been repeated.
                The purpose of this study was to determine the number and types of errors in nursing research journal reference lists.
                A stratified random sample of 262 references from three nursing research journals was obtained. References were compared with the actual articles, books, and chapters cited, or with photocopies obtained via interlibrary loan. Error rates were calculated.
                The overall error rate was 45.8%; 38.3% of all references contained at least one major error, and 13.8% of all references contained at least one minor error.
                The overall rate of reference errors falls within the range exhibited by recent studies of the medical and dental literature but exceeds the rates found in studies of nursing journals and veterinary medicine journals. Researchers need to consider the number and types of errors involved when using reference lists in their research. The section of a research article most likely to be read is the abstract, and therefore it is particularly important that the abstract reflect the article faithfully.
                To assess abstracts accompanying research articles published in 6 medical journals with respect to whether data in the abstract could be verified in the article itself.
                Analysis of simple random samples of 44 articles and their accompanying abstracts published during 1 year(July 1, 1996-June 30, 1997) in each of 5 major general medical journals (Annals of Internal Medicine, BMJ, JAMA, Lancet, and New England Journal of Medicine) and a consecutive sample of 44 articles published during 15 months (July 1, 1996-August 15, 1997) in the CMAJ.
                Abstracts were considered deficient if they contained data that were either inconsistent with corresponding data in the article's body (including tables and figures) or not found in the body at all.
                The proportion of deficient abstracts varied widely (18%-68%) and to a statistically significant degree (P<.001) among the 6 journals studied.
                Data in the abstract that are inconsistent with or absent from the article's body are common, even in large-circulation general medical journals. The quality of peer reviewing in developing countries is thought to be poor. To examine whether this was so, we compared the performance of Indian and non-Indian reviewers who were sent original and review articles submitted to The National Medical Journal of India. We also tested whether informing reviewers that their comments would be exchanged improved the quality of their reviews.
                In a prospective, randomized, blinded study, we sent 100 manuscripts to pairs of peer reviewers (Indian and non-Indian) of which 78 pairs of completed replies were available for analysis. Thirty-eight pairs of reviews were exchanged and 40 were not. The quality of the reviews was assessed by two editors who were unaware of the reviewers' nationality and whether they had been told that their reviews would be exchanged. The quality of the reviews was scored out of 100 (based on a predesigned evaluation proforma). We also measured the time taken to return a manuscript.
                Overall, non-Indian reviewers scored higher than Indians (mean scores non-Indians first, 56.7 v. 48.6, p < 0.001), especially those in the non-exchanged group (58.4 v. 47.3, p < 0.001) but not the exchanged group (54.8 v. 50.0, p < 0.06). Being informed that reviews would be exchanged did not affect the quality of reviews by non-Indians (54.8 exchanged v. 58.4 non-exchanged) or of reviews by Indians (50.0 exchanged v. 47.3 non-exchanged). The editors' assessment of the reviewers matched well (r = 0.59, p < 0.001). Non-Indians took the same amount of time as Indians to return their reviews, although the postage time was at least eight days longer.
                We found that non-Indian peer reviewers were better than Indians and informing them that their views would be exchanged did not seem to affect the quality of their reviews. We suggest that Indian editors should also use non-Indian reviewers and start training programmes to improve the quality of peer reviews in India. To compare the recall and precision of citation retrieval from MEDLINE when searching in journals containing traditional abstracts with journals containing abstracts prepared according to the guidelines of more informative abstracts (MIAs) for clinical journal articles.
                Analytic survey of operating characteristics of MeSH terms and textwords selected to detect studies meeting basic methodological criteria for direct clinical use in general adult medicine in priority journals with and without MIAs.
                The sensitivity, specificity, and precision of search terms were determined by comparison with a manual review of all articles (the gold standard) in 10 internal and general medicine journals for 1986 and 1991. The performance of each search term was compared before and after the implementation of MIAs and in two journal sets with and without MIAs in 1991.
                The sensitivity of some terms increased with the use of MIAs and also increased by the same amount in journals with usual abstracts in 1991 compared to 1986. The use of MIAs had the greatest impact in the areas of etiology and prognosis. Many MeSH and textwords had comparable performance before and after the implementation of MIAs and between journals with and without MIAs in 1991.
                More informative abstracts improved the retrieval properties of some, but not all, textwords and medical subject headings. Further work may be needed to enhance the effects of MIAs on bibliographic retrieval. Large clinical trials are the criterion standard for making treatment decisions, and nonpublication of the results of such trials can lead to bias in the literature and contribute to inappropriate medical decisions.
                To determine the rate of full publication of large randomized trials presented at annual meetings of the American Society of Clinical Oncology (ASCO), quantify bias against publishing nonsignificant results, and identify factors associated with time to publication.
                Survey of 510 abstracts from large (sample size, > or =200), phase 3, randomized controlled trials presented at ASCO meetings between 1989 and 1998. Trial results were classified as significant (P< or =.05 for the primary outcome measure) or nonsignificant (P>.05 or not reported), and the type of presentation and sponsorship were identified. Subsequent full publication was identified using a search of MEDLINE and EMBASE, completed November 1, 2001; the search was updated in November 2002, using the Cochrane Register of Controlled Trials. Authors were contacted if the searches did not find evidence of publication.
                Publication rate at 5 years; time from presentation to full publication.
                Of 510 randomized trials, 26% were not published in full within 5 years after presentation at the meeting. Eighty-one percent of the studies with significant results had been published by this time compared with 68% of the studies with nonsignificant results (P<.001). Studies with oral or plenary presentation were published sooner than those not presented (P =.002), and studies with pharmaceutical sponsorship were published sooner than studies with cooperative group sponsorship or studies for which sponsorship was not specified (P =.02). These factors remained significant in a multivariable model. The most frequent reason cited by authors for not publishing was lack of time, funds, or other resources.
                A substantial number of large phase 3 trials presented at an international oncology meeting remain unpublished 5 years after presentation. Bias against publishing nonsignificant results is a problem even for large randomized trials. Nonpublication breaks the contract that investigators make with trial participants, funding agencies, and ethics boards. To determine the accuracy of bibliographic citations in three widely circulated nursing journals, RN, Nursing Management, and Image: Journal of Nursing Scholarship. Accurate citations facilitate retrieval of the cited documents and establish the judgment and credibility of authors.
                Descriptive, comparative.
                A random sample of 60 references per journal published July 1995 to June 1996 was reviewed for citation and quotation accuracy. Errors were classified as major or minor. Data were analyzed using the SAS statistical package.
                Of the 180 references, 58 (32%) had citation errors, with 43 of 58 errors classified as major errors, which made retrieval of the cited work difficult; 12 of the 180 articles contained a major quotation error, including four instances where the content of the original article contradicted or was unrelated to the author's contention.
                The rate of citation and quotation errors in the three sampled nursing journals is comparable to rates previously reported for medical and nursing journals. Errors of citation and quotation diminish the value of published papers. Citation and quotation errors are common in medical journals. We assessed the prevalence of those errors in gross anatomy journals, where articles often cite old anatomical studies. The study included 199 randomly selected references from articles published in the first 2001 issue of three major gross anatomy journals: Annals of Anatomy, Clinical Anatomy, and Surgical and Radiologic Anatomy. The selected references were checked for accuracy against the original articles. Citation errors were classified as major, intermediate, and minor. Quotation errors were classified as major and minor. Citations errors were found in 27% (54/199) of the references and 38% of them were major errors. Errors occurred in 19% (52/272) of quotations and nearly all (94%) were major. Furthermore, 24% of the quotations were indirect references to a secondary, instead of original, source. There was no statistically significant difference in the rates of citation or quotation errors between the references published before or after the introduction of MEDLINE (chi2 test, P > 0.05) in 1963, and the prevalence of these errors in gross anatomy journals was similar to that found in other medical fields. A high proportion of major citation errors, a very high proportion of major quotation errors, and the substantial number of indirect quotations call for serious editorial action in anatomy journals.
                Copyright 2004 Wiley-Liss, Inc. Peer reviewers are blinded sometimes to authors' and institutions' names, but the effects of blinding on review quality are not known. We, therefore, conducted a randomized trial of blinded peer review. Each of 127 consecutive manuscripts of original research that were submitted to the Journal of General Internal Medicine were sent to two external reviewers, one of whom was randomly selected to receive a manuscript with the authors' and institutions' names removed. Reviewers were asked, but not required, to sign their reviews. Blinding was successful for 73% of reviewers. Quality of reviews was higher for the blinded manuscripts (3.5 vs 3.1 on a 5-point scale). Forty-three percent of reviewers signed their reviews, and blinding did not affect the proportion who signed. There was no association between signing and quality. Our study shows that, in our setting, blinding improves the quality of reviews and that research on the effects of peer review is possible. Improving precision and economy in the prescribing of drugs is a goal whose importance has increased with the proliferation of new and potent agents and with growing economic pressures to contain health-care costs. We implemented an office-based physician education program to reduce the excessive use of three drug groups: cerebral and peripheral vasodilators, an oral cephalosporin, and propoxyphene. A four-state sample of 435 prescribers of these drugs was identified through Medicaid records and randomly assigned to one of three groups. Physicians who were offered personal educational visits by clinical pharmacists along with a series of mailed "unadvertisements" reduced their prescribing of the target drugs by 14 per cent as compared with controls (P = 0.0001). A comparable reduction in the number of dollars reimbursed for these drugs was also seen between the two groups, resulting in substantial cost savings. No such change was seen in physicians who received mailed print materials only. The effect persisted for at least nine months after the start of the intervention, and no significant increase in the use of expensive substitute drugs was found. Academically based "detailing" may represent a useful and cost-effective way to improve the quality of drug-therapy decisions and reduce unnecessary expenditures. To evaluate the use and reporting of adjusted analysis in randomised controlled trials (RCTs) and compare the quality of reporting before and after the revision of the CONSORT Statement in 2001.
                Comparison of two cross sectional samples of published articles.
                Journal articles indexed on PubMed in December 2000 and December 2006.
                Parallel group RCTs with a full publication carried out in humans and published in English
                Proportion of articles reported adjusted analysis; use of adjusted analysis; the reason for adjustment; the method of adjustment and the reporting of adjusted analysis results in the main text and abstract.
                In both cohorts, 25% of studies reported adjusted analysis (84/355 in 2000 vs 113/422 in 2006). Compared with articles reporting only unadjusted analyses, articles that reported adjusted analyses were more likely to specify primary outcomes, involve multiple centers, perform stratified randomization, be published in general medical journals, and recruit larger sample sizes. In both years a minority of articles explained why and how covariates were selected for adjustment (20% to 30%). Almost all articles specified the statistical methods used for adjustment (99% in 2000 vs 100% in 2006) but only 5% and 10%, respectively, reported both adjusted and unadjusted results as recommended in the CONSORT guidelines.
                There was no evidence of change in the reporting of adjusted analysis results five years after the revision of the CONSORT Statement and only a few articles adhered fully to the CONSORT recommendations. A high rate of errors of citation and quotation has been reported in the publications of many medical specialties. The aim of this study was to determine the prevalence of citation and quotation errors in otolaryngology/head and neck surgery journals. A retrospective analysis was performed based on the first issue for 1997 of each of four journals: Laryngoscope; Annals of Otology, Rhinology and Laryngology; Clinical Otolaryngyology; and Journal of Laryngology and Otology. A sample of 50 references from each journal was randomly selected and each was checked for accuracy against the original referenced paper. Citation errors were categorized as major, intermediate or minor and quotation errors as major or minor. Citation errors occurred in 37.5% of the references, 11.9% of which were considered major errors. Quotation errors occurred in 17%, with 11.1% major errors. This prevalence is similar to the established error rate in medical literature. Citation errors are common among nursing journals. But, there are no data regarding the factors that predispose to these errors.
                To determine the risk factors that predispose to citation error in peer-reviewed nursing journals.
                Five hundred and fifty references were selected randomly from articles published in eleven nursing journals for the year 1998. The incidences of major and minor citation errors were determined by comparing with the original articles. The relative odds of citation errors for the number of authors, collaborating institutions and the length of the reference list were calculated. The correlation between the scientific quality of the journal (by means of journal impact factor and immediacy index) and the incidence of citation error were also determined.
                The incidence of citation errors is comparable to those reported previously. Long reference lists in articles written by a single author predicted strongly the occurrence of minor citation errors. Journals with a high impact factor and immediacy index tend to contain fewer minor mistakes. None of these factors affect the incidence of major errors.
                Contributors to journals should be aware of the various risk factors for citation errors. Citation accuracy may be improved by modifying these factors. This retrospective review of the 1990 ABA orally presented abstracts found an overall publication rate of slightly more than 26%. This included both same-content and related-content publications. When the abstracts were made a subset by theme, a major variance in the publication rates was found. Abstracts that had a psychologic content (72.73%), for example, fared much better than reports of in vitro research (6.9%). This overall rate when compared to publication studies in other clinical areas was found to be quite low. This study further found that there was a great deal of similarity between abstracts that were published and those that were not. The one meaningful predictor of ultimate journal publication was the clarity of the presentation of statistically related information. Sixty research investigations published in the biomedical literature were analyzed to examine the effect of design attributes on outcome. All 60 studies included a controlled trial involving a pretest, a therapeutic intervention, and a posttest across at least two groups. Thirty of the trials used random assignment of participants to treatment or control conditions and 30 trials employed some nonrandom method of subject assignment. Trial results were aggregated and evaluated by comparing effect sizes for the primary statistical test of the hypothesis. Data analysis revealed that the trial results, as measured by effect size, did not vary across therapeutic trials using random assignment versus those using nonrandom allocation of subjects. The impact of design attributes in the interpretation of multiple clinical trials addressing a similar research question is examined. The argument is made that various design attributes frequently associated with methodological quality should be considered as important moderator variables and their influence on trial outcome should not be assumed a priori but rather examined empirically. To characterize the structured abstracts in biomedical journals indexed in MEDLINE over a three-year period as an initial step in exploring their utility in enhancing bibliographic retrieval.
                The study examined the occurrence of structured abstracts in MEDLINE from March 1989 to December 1991, characteristics of MEDLINE records for articles with structured abstracts, editorial policies of six selected MEDLINE journals on structured abstracts, and a sample of twenty-five structured abstracts from the six journals.
                The study revealed that the number of structured abstracts in MEDLINE and the number of MEDLINE journals publishing structured abstracts increased substantially between 1989 and 1991. On average, articles with structured abstracts had more access points (Medical Subject Heading [MeSH] terms and text words) than MEDLINE articles as a whole. The average length of the structured abstract was greater than the average length of all abstracts in MEDLINE.
                The presence of structured abstracts may be associated with other article characteristics that lead to the assignment of a higher average number of MeSH headings or may itself contribute to the assignment of more headings. The variations in the structured-abstract formats prescribed by different journals may complicate the exploitation of these abstracts in bibliographic retrieval systems. More research is needed on a number of questions related to the quality and utility of structured abstracts. Little research has been conducted into the quality of peer review and, in particular, the effects of blinding peer reviewers to authors' identities or masking peer reviewers' identities.
                To determine whether concealing authors' identities from reviewers (blinding) and/or revealing the reviewer's identity to a coreviewer (unmasking) affects the quality of reviews, the time taken to carry out reviews, and the recommendation regarding publication.
                Randomized trial of 527 consecutive manuscripts submitted to BMJ, which were randomized and each sent to 2 peer reviewers.
                Manuscripts were randomized as to whether the reviewers were unmasked, masked, or uninformed that a study was taking place. Two reviewers for each manuscript were randomized to receive either a blinded or an unblinded version.
                Mean total quality score, time taken to carry out the review, and recommendation regarding publication.
                Of the 527 manuscripts entered into the study, 467 (89%) were successfully randomized and followed up. The mean total quality score was 2.87. There was little or no difference in review quality between the masked and unmasked groups (scores of 2.82 and 2.96, respectively) and between the blinded and unblinded groups (scores of 2.87 and 2.90, respectively). There was no apparent Hawthorne effect. There was also no significant difference between groups in the recommendations regarding publication or time taken to review.
                Blinding and unmasking made no editorially significant difference to review quality, reviewers' recommendations, or time taken to review. Other considerations should guide decisions as to the form of peer review adopted by a journal, and improvements in the quality of peer review should be sought via other means. The aim of the study was to investigate the reference and quotation accuracy in four peer-reviewed manual therapy journals. A stratified random sample of original research (n=7) was collected from each of the journals spanning the years January 2000 to December 2001. A further random selection of 80 references from each journal paper sampled was then reviewed (Total N=320) for citation and quotation accuracy. Numbers of citations with errors were determined, then classified as either major or minor and categorized by bibliographic headings (author, title, journal, year, volume, page and irretrievable). Each quotation was individually assessed for accuracy and judged to be either correct or incorrect. A quotation was deemed correct if it accurately substantiated and reported the original authors assertions. One hundred and fifteen citations across all journals contained errors (35.9%). Some citations exhibited multiple major and minor errors. Bibliographically classified errors for all journals showed 61 author, 51 title, 6 journal, 4 year, 12 volume and 25 page errors. JMPT showed the lowest referencing error rate (20%) while JBWMT recorded the highest (58.8%). The total number of quotation errors across all journals was 69 (12.3%). JMPT showed the lowest quotation error rate of 6 (4.7%), MT had 12 errors (7.3%), JOM produced 21 errors (13.3%), while JBWMT recorded the highest error rate with 32 (27.6%). Poor citation and quotation is a reflection on the scholarly work of the authors and the journal. The trend for errors in quotation is more worrying than citation errors as it reflects poor diligence on the part of the investigators. To assess and compare the quality of nonstructured and structured abstracts of original research articles in three medical journals.
                Blind, criterion-based observational study.
                Random sample of 300 abstracts (25 abstracts per journal each year) of articles published in the British Medical Journal (BMJ), the Canadian Medical Association Journal and the Journal of the American Medical Association (JAMA) in 1988 and 1989 (nonstructured abstracts) and in 1991 and 1992 (structured abstracts).
                The quality of abstracts was measured against 33 objective criteria, which were divided into eight categories (purpose, research design, setting, subjects, intervention, measurement of variables, results and conclusions). The quality score was determined by dividing the number of criteria present by the number applicable; the score varied from 0 to 1.
                The overall mean quality scores for nonstructured and structured abstracts were 0.57 and 0.74 respectively (p < 0.001). The frequency in meeting the specific criteria was generally higher for the structured abstracts than for the nonstructured ones. The mean quality score was higher for nonstructured abstracts in JAMA than for those in BMJ (0.60 v. 0.54, p < 0.05). The scores for structured abstracts did not differ significantly between the three journals.
                The findings support recommendations that promote the use of structured abstracts. Further studies should be performed to assess the effect of time on the quality of abstracts and the extent to which abstracts reflect the content of the articles. Previous studies have found high error rates in references in biomedical journals. The aim of this paper was to assess the accuracy of references in three Australian and New Zealand general medical journals.
                References from the August 1999 issues of the Medical Journal of Australia, the New Zealand Medical Journal and the Australian and New Zealand Journal of Medicine were assessed for accuracy using PubMed of the National Library of Medicine.
                This study found a high rate of reference errors in Australian and New Zealand medical journals. The reference error rate ranged from 33.5 to 48.8%. The most frequent errors were in the author's names and in the title.
                The reference error rate in Australia and New Zealand medical journals is high and is preventable. Authors should be more diligent and preferably verify cited references against the original article. The accuracy of references was assessed in Turkish journals of orthopedics and traumatology.
                All references cited in four journals (Acta Orthopaedica et Traumatologica Turcica, Arthroplasty Arthroscopic Surgery, Hacettepe Journal of Orthopaedic Surgery, and The Journal of Turkish Spinal Surgery) throughout 1995 to 1999 were numbered consecutively. One percent of references from each journal was randomly selected to be evaluated in comparison with primary sources. Errors were classified in three categories. Another randomized selection was made to check 50 references from each journal. The results were assessed by the t-test.
                Of 18,193 references cited in four journals, 182 references were checked. Errors were found in 117 references (63%), seven of which were major errors that prevented location of the primary source. When 50 references were checked from each journal, the mean inaccuracy rate was 60%. No significant differences were found between the journals with respect to the frequency of erroneous references.
                The high rate of inaccuracy emphasizes the need for greater care in the use of references on the part of authors, journal editors, and reviewers. Research projects are frequently presented at national meetings not only to make the data available, but also to further critically evaluate the project. The ultimate goal remains publication of the research. We assessed the publication rate of presentations at 1993-95 annual meetings.
                All presentations at the scientific sessions were searched in the computerized database Melvyl MEDLINE and PubMed. The papers were categorized in 6 broad groups.
                The overall proportion of presentations that were published was 32%. Clinical papers in pediatrics had the highest publication rate, followed by basic science research in laryngology and plastics.
                The publication rate of 32% is lower than the rate at meetings in other fields, reflecting variability in selection criteria for the presentations and quality of abstracts presented. Basic science presentations did not have a greater publication rate as compared to clinical presentations. A study was designed to test the hypothesis that experts who review papers for publication are prejudiced against an unconventional form of therapy. Two versions were produced (A and B) of a 'short report' that related to treatments of obesity, identical except for the nature of the intervention. Version A related to an orthodox treatment, version B to an unconventional treatment. 398 reviewers were randomized to receive one or the other version for peer review. The primary outcomes were the reviewers' rating of 'importance' on a scale of 1-5 and their verdict regarding rejection or acceptance of the paper. Reviewers were unaware that they were taking part in a study. The overall response rate was 41.7%, and 141 assessment forms were suitable for statistical evaluation. After dichotomization of the rating scale, a significant difference in favour of the orthodox version with an odds ratio of 3.01 (95% confidence interval, 1.03 to 8.25), was found. This observation mirrored that of the visual analogue scale for which the respective medians and interquartile ranges were 67% (51% to 78.5%) for version A and 57% (29.7% to 72.6%) for version B. Reviewers showed a wide range of responses to both versions of the paper, with a significant bias in favour of the orthodox version. Authors of technically good unconventional papers may therefore be at a disadvantage in the peer review process. Yet the effect is probably too small to preclude publication of their work in peer-reviewed orthodox journals. The authors investigated the fate of abstracts from each ARVO section (May 1985 meeting), the overall publication percentage, and the journals in which the abstract-derived articles were published.
                They performed a MEDLINE search by first author for 25 or 26 randomly selected abstracts from each section to identify those that led to full-length articles in peer-reviewed journals.
                Overall, 63% of abstracts led to full-length articles in peer-reviewed journals within our search period of 87 months. The publication rate of oral presentation abstracts (68%) was significantly higher than that of poster presentation abstracts (56%). A greater proportion of basic science-oriented abstracts (67%) led to publication than the clinically oriented abstracts (56%). The rate of publication was lowest for the Cornea section (40%) and highest for Physiology and Pharmacology (80%) and Biochemistry (76%). The abstract-derived articles were published in 67 different peer-reviewed journals, with 43% of the articles appearing in only five journals.
                The fact that the majority of abstracts led to full-length articles supports ARVO's goal of a large interdisciplinary appeal with the exchange of ideas among different investigators. Over the past decade, on-line databases have become increasingly popular among health care professionals. As a group, these 'end-users' report utilizing databases to keep abreast of medical progress, to conduct research and to address specific patient care issues. Throughout the literature, medical professionals ('content experts') have proved to be less effective searchers than librarians ('search experts'). The potential implications of this discrepancy are worrysome. For any given clinical scenario, for example, published reports may reach contradictory conclusions. A poorly skilled searcher may not retrieve enough articles to appreciate this fact. Optimizing searching skills is therefore a worthwhile goal. As a first step, many medical schools introduce students to on-line databases, most notably MEDLINE. Residency is an ideal time to continue this training. A recognized obstacle to provide residents with formal MEDLINE instruction is time constraint. We therefore conducted this study to ascertain the impact an individual 1-hour tutorial session would have on MEDLINE utilization among obstetrics and gynecology residents training at an academic medical centre. Outcome measures included MEDLINE search frequency, duration, recall, precision and searcher satisfaction. Search recall measures the searcher's ability to retrieve articles deemed relevant to the question at hand. Search precision gauges the searchers' ability to eliminate irrelevant articles. Although the sessions were well received, we were unable to demonstrate an improvement in the outcome measures analysed. Further research is therefore indicated so that cost-effective educational strategies can be recommended for wide-scale use. The accuracy of quotations and references in six medical journals published during January 1984 was assessed. The original author was misquoted in 15% of all references, and most of the errors would have misled readers. Errors in citation of references occurred in 24%, of which 8% were major errors--that is, they prevented immediate identification of the source of the reference. Inaccurate quotations and citations are displeasing for the original author, misleading for the reader, and mean that untruths become "accepted fact." Some suggestions for reducing these high levels of inaccuracy are that papers scheduled for publication with errors of citation should be returned to the author and checked completely and a permanent column specifically for misquotations could be inserted into the journal. The purpose of this study was to review abstracts accepted for presentation at the annual national meeting of the Society for Adolescent Medicine (SAM) over the past 3 decades for subject content and research design.
                Retrospective review was conducted of research abstracts for 3 years, selected at random, for the 1990s and then matched at 10-year intervals for the previous 2 decades. The major categories for subject content were: medical, psychosocial, health services, and miscellaneous; for research design categories were: retrospective, observational, experimental, and miscellaneous. Subsequent publication was also assessed.
                The most notable change over the 3 decades in subject content was a shift from medical topics (58% to 19%) to psychosocial topics [20% to 52% (p < .001)] of abstracts, the latter largely accounted for by increases in topics pertaining to high-risk behaviors. There was a substantial decrease in abstracts concerning medical aspects of chronic illness (p < .001), well-adolescent care, growth and development, and psychosocial aspects of chronic illness (p < .02). Regarding research design, the proportion of retrospective studies decreased over the 3 decades (p < .02); the most prevalent design was observational, with an increase in studies using established databases (p < .02). Less than 15% represented an experimental design. Studies using qualitative analysis appeared for the first time in the 1990s. The percentage of presentations which eventuated in published reports increased from 35% to 63% over the 3 decades (p < .02).
                A significant shift occurred in subject content of scientific abstracts from a medical to a psychosocial emphasis. The proportion of retrospective designs decreased, and qualitative analyses emerged. The likelihood of later publication of the studies increased. Literature references contain a considerable number of inaccuracies, both in citations and in quotations. We investigated the accuracy of quotations and references in Nederlands Tijdschrift voor Geneeskunde. We checked at random 100 references of articles presented for publication during a period of 4 weeks using Index Medicus. The citation and quotation of another 100 references (50 with a reading list of less than 13, and 50 with a reading list of more than 24 articles) of published original articles were checked, using the original article. In the references of articles presented for publication 70% contained one or more inaccuracies. In the published articles 31% was not completely correct. In 5% these errors prevented immediate identification of the cited article. In one of every three articles a citation error was made: 5 trivial errors, 16 slightly misleading and 12 seriously misleading errors were found. We could not find a relation between the number of citation errors and the length of the reading list. The original source was not cited in 8 articles. Our conclusion is that the number of inaccuracies, in references as well as in citations, is high; authors should check this part of their articles more carefully. Notably the use of multiple references after a sentence with a number of statements is to be avoided. Many international journals require authors of randomized controlled trials to adhere to standards of reporting described in the statement of the Consolidated Standards of Reporting Trials (CONSORT) and the requirements issued by the International Committee of Medical Journal Editors (ICMJE). We examined the extent to which these international standards have been adopted by Indian medical journals.
                To identify Indian medical journals that publish randomized controlled trials, we did electronic searches of the websites of the National Informatics Centre (Indian Medlars Centre), Database of Open Access Journals, National Library of Medicine, WHO's Index Medicus for the South-East Asian region and Google. We analysed their instructions to authors for endorsement of the CONSORT statement and the ICMJE requirements for reporting of randomized controlled trials. We then identified all randomized controlled trials published in these journals during 2004 and 2005 and assessed them against selected CONSORT items and ICMJE requirements, and scored them on the Jadad scale.
                Of the 65 journals selected, 38 (58.5%) mentioned the ICMJE requirements in their instructions for authors but only 20 (31%) specifically required authors to submit manuscripts in accordance with the CONSORT statement. Of 151 randomized controlled trials published during 2004-05, only 4 of 13 (30.8%) selected CONSORT items were reported in > 50% of trial reports. Items reflecting internal validity were poorly reported. Jadad scores were significantly higher for general medical journals compared with specialty journals (mean difference 0.46; 95% CI: 0.15-0.78; p = 0.005) and in trials published in 2005 over those published in 2004 (mean difference 0.48; 95% CI: 0.18-0.79; p = 0.002). Ethical issues were poorly reported in one-third of reports, and sources of funding and conflicts of interest were not declared in over three-fourths. Adequacy of reporting was not related to endorsing either the CONSORT statement or the ICMJE requirements.
                Medical journals published in India should adopt internationally recognized norms for reporting clinical trials and work with authors, reviewers and institutional review boards to improve the standards of conduct, reporting and validity of inferences of trials. To explore the impact of methodologic issues on the results of meta-analyses. The following issues were examined: the type of literature search strategy used; inclusion or exclusion of non-peer-reviewed studies; the inclusion or exclusion of non-English language publications; the effect of trial quality; and the inclusion or exclusion of non-placebo-controlled studies.
                The International Study of Perioperative Transfusion (ISPOT) meta-analyses were used to evaluate each of the methodologic issues. The 10 meta-analyses consisted of technologies to reduce the need for perioperative red blood cell transfusion. The number of trials for each of the meta-analyses varied from 2 to 45. Both EMBASE and MEDLINE searches were conducted, including the use of systematic search strategies.
                MEDLINE identified the vast majority of trials. Alone, MEDLINE would have missed 8 studies compared to 10 for EMBASE. Use of the systematic search strategies greatly reduced the number of articles to be reviewed compared to open searches. Type of publication, country of study origin, inclusion of non-English publications, and trial quality had very little impact on the estimates of effect. The use of placebo versus open-label control affected the magnitude of the odds ratio for two of the meta-analyses. The results of the two meta-analyses were not statistically significant if only placebo-controlled trials were included.
                While methodologic issues had very little impact on the ISPOT meta-analyses, further studies are needed in a variety of other clinical settings. Because MEDLINE, coupled with a review of the references in the identified trials, identified the vast majority of trials, one needs to consider the costs and benefits of searching EMBASE and the pursuance of unpublished and unindexed trials. Studies of drug treatments are more likely to report favourable outcomes when they are funded by the pharmaceutical industry. We compared drug trials reported in three major psychiatric journals to investigate these influences. Independent studies were more likely to report negative findings than industry-funded studies. However, the involvement of a drug company employee had a much greater effect on study outcome than financial sponsorship alone. Studies with positive results are more likely to be published in biomedical journals than are studies with negative results. However, many studies submitted for consideration at scientific meetings are never published in full; bias in this setting is poorly studied.
                To identify features associated with the fate of research abstracts submitted to a scientific meeting.
                Prospective observational cohort, with 5-year follow-up of all research submitted for consideration to the major annual 1991 US research meeting in the specialty of emergency medicine.
                All research abstracts submitted for consideration at the meeting for possible presentation.
                Characteristics associated with acceptance for presentation at the meeting and subsequent publication as a full manuscript.
                A total of 492 research abstracts were submitted from programs in emergency medicine and other specialties affiliated with 103 US medical schools. A total of 179 (36%) were accepted for presentation and 214 (43%) were published in 44 journals. Of the 179 abstracts accepted for presentation, 111 studies were published. Scientific quality of abstracts or prestige of the journal in which the study was eventually published did not predict either of these outcomes. The best predictors (by logistic regression) of meeting acceptance were a subjective "originality" factor (odds ratio [OR], 2.07; 95% confidence interval [CI], 1.13-3.89) and positive results (OR, 1.99; 95% CI, 1.07-3.84), and, for publication, meeting acceptance (OR, 2.49; 95% CI, 1.49-4.35) and large sample size (OR, 2.26; 95% CI, 1.23-4.31). Forty-nine percent (241) of abstracts did not report on blinding, and 24% (118) did not report on randomization. Acceptance and publication were both more likely for positive outcomes (P=.03). Funnel plots showed the classic distribution of positive-outcome ("publication") bias at each of the submission, acceptance, and publication phases. Meeting acceptance predicted publication with a sensitivity of only 51%, specificity of 71%, positive predictive value of 57%, and negative predictive value of 66%.
                Positive-outcome bias was evident when studies were submitted for consideration and was amplified in the selection of abstracts for both presentation and publication, neither of which was strongly related to study design or quality. --To investigate factors associated with the publication of research findings, in particular, the association between "significant" results and publication.
                --Follow-up study.
                --Studies approved in 1980 or prior to 1980 by the two institutional review boards that serve The Johns Hopkins Health Institutions--one that serves the School of Medicine and Hospital and the other that serves the School of Hygiene and Public Health.
                --A total of 737 studies were followed up.
                --Of the studies for which analyses had been reported as having been performed at the time of interview, 81% from the School of Medicine and Hospital and 66% from the School of Hygiene and Public Health had been published. Publication was not associated with sample size, presence of a comparison group, or type of study (eg, observational study vs clinical trial). External funding and multiple data collection sites were positively associated with publication. There was evidence of publication bias in that for both institutional review boards there was an association between results reported to be significant and publication (adjusted odds ratio, 2.54; 95% confidence interval, 1.63 to 3.94). Contrary to popular opinion, publication bias originates primarily with investigators, not journal editors: only six of the 124 studies not published were reported to have been rejected for publication.
                --There is a statistically significant association between significant results and publication. The high rate of citation errors in bibliographies in medical journals has been a source of concern in recent years. We examined the accuracy of references published during 1 year in two Israeli medical journals. Only two-thirds of the randomly selected references examined were error free; 8% had major errors preventing identification of the cited article. Most of the errors found (76%) were in referencing the author(s) or title of the article. We conclude that errors in citation appear also in Israeli medical journals. Editors should emphasize the importance of reference verification; however, primary responsibility for the accuracy of the reference list rests with the author. Authors should exercise more care in preparing bibliographies and should invest more effort in verification of quoted references. To explore whether searching specialised bibliographic databases identified additional relevant papers to those located by a Medline search for a systematic review of exercise therapy.
                Searches were performed in Medline, two further generalised medical databases (Embase, Cochrane Library) and four specialised databases (CancerLit, Cinahl, PsychInfo, SportDiscus) to identify controlled trials of exercise interventions for cancer patients.
                A total of 749 different publications were located through the search, of which 18 met inclusion criteria. Fifteen (83%) of these were identified through Medline and three (17%) from three individual specialised databases. A further seven studies meeting inclusion criteria were located through reference lists and contact with experts.
                In this example, searching Medline and additional specialised databases along with checking reference lists and contacting experts was the most effective means of ensuring that all relevant papers were included in the review. Searching Medline alone for systematic reviews of exercise or other unconventional therapies is likely to be inadequate. Significant resources and time are invested in the production of research knowledge. The primary objective of this randomized controlled trial was to evaluate the effectiveness of three knowledge translation and exchange strategies in the incorporation of research evidence into public health policies and programs.
                This trial was conducted with a national sample of public health departments in Canada from 2004 to 2006. The three interventions, implemented over one year in 2005, included access to an online registry of research evidence; tailored messaging; and a knowledge broker. The primary outcome assessed the extent to which research evidence was used in a recent program decision, and the secondary outcome measured the change in the sum of evidence-informed healthy body weight promotion policies or programs being delivered at health departments. Mixed-effects models were used to test the hypotheses.
                One hundred and eight of 141 (77%) health departments participated in this study. No significant effect of the intervention was observed for primary outcome (p < 0.45). However, for public health policies and programs (HPPs), a significant effect of the intervention was observed only for tailored, targeted messages (p < 0.01). The treatment effect was moderated by organizational research culture (e.g., value placed on research evidence in decision making).
                The results of this study suggest that under certain conditions tailored, targeted messages are more effective than knowledge brokering and access to an online registry of research evidence. Greater emphasis on the identification of organizational factors is needed in order to implement strategies that best meet the needs of individual organizations.
                The trial registration number and title are as follows: ISRCTN35240937 -- Is a knowledge broker more effective than other strategies in promoting evidence-based physical activity and healthy body weight programming? To investigate the relationship between industry- vs nonindustry-funded publications comparing the efficacy of topical prostaglandin analogs by evaluating the correspondence between the statistical significance of the publication's main outcome measure and its abstract conclusions.
                Retrospective, observational cohort study.
                English publications comparing the ocular hypotensive efficacy between any or all of latanoprost, travoprost, and bimatoprost were searched from the MEDLINE database. Each article was reviewed by three independent observers and was evaluated for source of funding, study quality, statistically significant main outcome measure, correspondence between results of main outcome measure and abstract conclusion, number of intraocular pressure outcomes compared, and journal impact factor. Funding was determined by published disclosure or, in cases of no documented disclosure, the corresponding author was contacted directly to confirm industry funding. Discrepancies were resolved by consensus. The main outcome measure was correspondence between abstract conclusion and reported statistical significance of the publications' main outcome measure.
                Thirty-nine publications were included, of which 29 were industry funded and 10 were nonindustry funded. The published abstract conclusion was not consistent with the results of the main outcome measure in 18 (62%) of 29 of the industry-funded studies compared with zero (0%) of 10 of the nonindustry-funded studies (P = .0006). Twenty-six (90%) of the industry-funded studies had proindustry abstract conclusions.
                Twenty-four percent of the industry-funded publications had a statistically significant main outcome measure; however, 90% of the industry-funded studies had proindustry abstract conclusions. Both readers and reviewers should scrutinize publications carefully to ensure that data support the authors' conclusions. To evaluate the sensitivity and precision of various extended search methods in identifying randomized controlled trials (RCTs) for systematic reviews.
                Prospective analysis of extended search methods (specialized databases or trial registries, reference lists, hand-searching, personal communication, and Internet) used in two systematic reviews of RCTs. The gold standard was the total number of RCTs identified by major databases (MEDLINE, EMBASE, etc.) and extended search strategies combined. Sensitivity was the proportion of all known RCTs identified by any extended search method. Precision reflected the proportion of all items uncovered by any extended search method that actually were RCTs.
                The extended search identified 94 additional RCTs for the systematic reviews beyond those identified with the major databases. Specialized databases and trial registries had the highest sensitivity and precision for the lipid-lowering project (13.6% and 52.7%, respectively; p < .05) followed by scanning of reference lists (7.2% sensitivity and 41.9% precision; p <.05). Hand-searching was more effective than personal communication and Internet searching (1.7% sensitivity and 12.2% precision; p < .05). The acupuncture project had slightly different results, with the specialized databases and trial registries tied with the review of reference lists for highest sensitivity (14.2%). The precision followed the same trend as the lipid-lowering project (17.6% specialized databases; 8.3% reference lists; p < .05). A post-hoc analysis showed that 75 of the 94 RCTs were indexed in the major databases but missed by the major database search.
                Extended searching identified additional RCTs for the systematic reviews beyond those found in major databases. Specialized databases and trial registries were most effective. An important number of RCTs were missed by the major database search. Timing and accuracy of indexing may explain this finding. The definitive measure, whether there is an association between the method used to uncover RCTs, the quality of the items uncovered and their impact on systematic review results, is yet to be determined. To determine whether alternative methods of presenting a contrast between the same two quantities in descriptions of research results could lead to different treatment decisions by physicians.
                We conducted a survey of practicing physicians and of faculty and fellows in training programs in clinical epidemiology and social science research methods. Each questionnaire presented results from a published study of either hypertension or hypercholesterolemia in two different ways: once as the relative change in the outcome rate and once as the absolute change in the outcome rate. We asked respondents to read each summary and indicate how the information contained in the summary would influence decisions about treatment.
                Of the 235 physicians who completed the questionnaire, 108 (46%) gave different responses to the same results presented in different ways. Of these, 97 (89.8%) indicated a stronger inclination to treat patients after reading of the relative change in the outcome rate (p less than 0.0001).
                The manner of presentation of results can influence physicians' judgments about the treatment of patients. We compared the quality of structured abstracts of original research articles from the British Medical Journal (BMJ), Canadian Medical Association Journal (CMAJ), and the Journal of the American Medical Association (JAMA) from 1991 to 1992 and 2001 to 2002 between journals.
                A random, stratified sample of 54 abstracts from 2001 to 2002 in BMJ, CMAJ, and JAMA was compiled and coded. Two blinded raters reviewed 27 abstracts each against 33 objective criteria, separated into eight categories (purpose, research design, setting, subjects, intervention, measurement of variables, results, and conclusion). The quality score was the proportion of criteria present (range = 0-1).
                The overall mean quality score (0.74) for 2001-2002 was significantly higher than the 1988-1989 unstructured abstracts (mean = 0.57; p<0.001) but not different from the 1991-1992 structured abstracts (mean = 0.74; p>0.05). In 2001-2002, abstracts of CMAJ and JAMA (both means = 0.76) improved significantly over 1991-1992 (p<0.05) and scored significantly higher than BMJ (mean = 0.71; d.f. = 16, p<0.05). Some individual criteria scores (intervention, statistical information) improved but information was found consistently under-represented in areas that imply shortcomings of the studies.
                We found a consistency in abstract quality regardless of the precise format used by different journals. This indicates that the framework for research articles already in place should be maintained and further modification of the framework may not necessarily improve the abstract quality. To examine the effect on peer review of asking reviewers to have their identity revealed to the authors of the paper.
                Randomised trial. Consecutive eligible papers were sent to two reviewers who were randomised to have their identity revealed to the authors or to remain anonymous. Editors and authors were blind to the intervention.
                The quality of the reviews was independently rated by two editors and the corresponding author using a validated instrument. Additional outcomes were the time taken to complete the review and the recommendation regarding publication. A questionnaire survey was undertaken of the authors of a cohort of manuscripts submitted for publication to find out their views on open peer review.
                Two editors' assessments were obtained for 113 out of 125 manuscripts, and the corresponding author's assessment was obtained for 105. Reviewers randomised to be asked to be identified were 12% (95% confidence interval 0.2% to 24%) more likely to decline to review than reviewers randomised to remain anonymous (35% v 23%). There was no significant difference in quality (scored on a scale of 1 to 5) between anonymous reviewers (3.06 (SD 0.72)) and identified reviewers (3.09 (0.68)) (P=0.68, 95% confidence interval for difference - 0.19 to 0.12), and no significant difference in the recommendation regarding publication or time taken to review the paper. The editors' quality score for reviews (3.05 (SD 0.70)) was significantly higher than that of authors (2.90 (0.87)) (P<0.005, 95%confidence interval for difference - 0.26 to - 0.03). Most authors were in favour of open peer review.
                Asking reviewers to consent to being identified to the author had no important effect on the quality of the review, the recommendation regarding publication, or the time taken to review, but it significantly increased the likelihood of reviewers declining to review. Research abstracts are frequently referenced in orthopaedic textbooks and influence orthopaedic care. However, little is known about the quality of information provided in the abstracts, the frequency of publication of complete papers after presentation of abstracts, or any discrepancies between abstracts and published papers. The objective of this study was to determine the quality of information provided in orthopaedic abstracts, rates of publication of full-text articles after presentation of abstracts, predictors of publication of full-text articles, and consistency between abstracts and full-text articles.
                We retrieved all abstracts from the 1996 scientific program of the sixty-third Annual Meeting of the American Academy of Orthopaedic Surgeons. For each abstract, we recorded the completeness of reporting and key features of the study design, conduct, analysis, and interpretation. A computerized Medline and PubMed search established whether the abstract had been followed by publication of a full-text article. Finally, we evaluated the consistency of reporting between abstracts and final publications.
                The program included 465 abstracts, 66% of which were on prognostic studies. All abstracts described the study design, and 70.7% of the designs were observational. Key methodological issues were reported in less than half of the abstracts, and information on data analysis was reported in <15%. One hundred and fifty-nine (34%) of the 465 abstracts were followed by publication of a full-text article. The mean time to publication (and standard deviation) was 17.6 +/- 12 months (range, one to fifty-six months). Inconsistencies between the abstract and the full-text article included the primary outcome measure, which differed 14% of the time, and the results, which differed 19% of the time.
                Two-thirds of the orthopaedic abstracts in this sample were not followed by publication of a full-text paper. The overall quality of reporting in abstracts proved inadequate, and inconsistencies between the final published paper and the original abstract occurred frequently. The routine use of abstracts as a guide to orthopaedic practice needs to be reconsidered. To assess the reporting quality of randomized controlled trials (RCTs) in the primary treatment of brain tumors and to identify significant predictors of quality.
                Two investigators searched MEDLINE, EMBASE, and bibliographies of retrieved articles for RCTs in the primary treatment of brain tumors published between January 1990 and December 2004. We assessed the quality of overall reporting and key methodologic factors reporting (allocation concealment, blinding, and intention to treat [ITT]). Two investigators also rated articles independently using items from the revised Consolidated Standards of Reporting Trials statement. A generalized estimated equation was used to generate regression models that identified significant factors associated with quality of reporting.
                We retrieved 74 relevant RCTs that randomly assigned 14,498 brain tumor patients. The quality of overall reporting has improved during the last 15 years, but eight of the 15 methodologic items were reported in less than 50% of trials. In the appraisal of the reporting quality of key methodologies, allocation concealment, blinding, and adherence to the ITT principle were reported in less than 30% of articles. Multivariable regression models revealed that an impact factor more than 1.66, publication after 1995, and sample size more than 280 were significant factors associated with better overall reporting, whereas complete industrial funding, impact factors more than 2.64, and positive primary outcomes were predictors of higher ratings of the three most important methodologic qualities.
                Despite improvement in general reporting quality, key methodologies that safeguard against biases may still benefit from better description. Significant factors associated with better reporting may act as surrogates for other characteristics. The objective of this study was to compare the performance of MEDLINE and EMBASE for the identification of articles regarding controlled clinical trials (CCTs) published in English and related to selected topics: rheumatoid arthritis (RA), osteoporosis (OP), and low back pain (LBP). MEDLINE and EMBASE were searched for literature published in 1988 and 1994. The initial selection of papers was then reviewed to confirm that the articles were about CCTs and to assess the quality of the studies. Selected journals were also hand searched to identify CCTs not retrieved by either database. Overall, 4111 different references were retrieved (2253 for RA, 978 for OP, and 880 for LBP); 3418 (83%) of the papers were in English. EMBASE retrieved 78% more references than MEDLINE (2895 versus 1625). Overall, 1217 (30%) of the papers were retrieved by both databases. Two hundred forty-three papers were about CCTs. Two-thirds of these were retrieved by both databases, and one-third by only one. An additional 16 CCTs not retrieved by either database were identified through hand searching. Taking these into account, EMBASE retrieved 16% more CCTs than MEDLINE (220 versus 188); the EMBASE search identified 85% of the CCTs compared to 73% by MEDLINE. No significant differences were observed in the mean quality scores and sample size of the CCTs missed by MEDLINE compared to those missed by EMBASE. Our findings suggest that the use of MEDLINE alone to identify CCTs is inadequate. The use of two or more databases and hand searching of selected journals are needed to perform a comprehensive search. To evaluate and compare the effects of two types of continuing medical education (CME) programs on the communication skills of practicing primary care physicians.
                Fifty-three community-based general internists and family practitioners practicing in the Portland, Oregon, metropolitan area and 473 of their patients.
                For the short program (a 4 1/2-hour workshop), 31 physicians were randomized to either the intervention or the control group. In the long program (a 2 1/2-day course), 20 physicians participated with no randomization. A research assistant visited all physicians' offices both one month before and one month after the CME program and audiotaped five sequential visits each time. Data were based on analysis of the content and the affect of the interviews, using the Roter Interactional Analysis Scheme.
                Based on both t-test analysis and analysis of covariance, no effect on communication was evident from the short program. The physicians enrolled in the long program asked more open-ended questions, more frequently asked patients' opinions, and gave more biomedical information than did the physicians in the short program. Patients of the physicians who attended the long program tended to disclose more biomedical and psychosocial information to their physicians. In addition, there was a decrease in negative affect for both patient and physician, and patients tended to demonstrate fewer signs of outward distress during the visit.
                This study demonstrates some potentially important changes in physicians' and patients' communication after a 2 1/2-day CME program. The changes demonstrated in both content and affect may have important influences on both biologic outcome and physician and patient satisfaction. The accuracy of the reference list is considered an essential factor in determining the quality of a scientific publication, and has been reviewed and published in several medical journals. The rate of citation errors in NEFROLOGIA has not been published before. The aim of this study has been to examine the accuracy of the references citation in the numbers of the journal published for a period of 15 years, from the first issue in 1981 to the last one on 1995.
                In this period 68 ordinary issues of the journal were published, with 520 original articles that constituted the data base, 87 articles were selected applying a table of random numbers with 5 references of each article selected to be reviewed with a total of 435. The accuracy of the name of the authors, title of the article, volume, initial and last page and year of publication were evaluated, accordingly with the NEFROLOGIA normative. Citation errors were considered major or minor depending on whether they prevented or not the immediate retrieval of the article listed.
                Of the 435 citations examined, 433 could be located, being impossible in 2 with the data supplied. In 189 (43.45%) 237 errors were detected, with 13.50% of them being considered major and 86.50% minor, with the greatest percentage of errors occurring in the data about the authors of the articles referenced 96 (40.50%).
                The issues of NEFROLOGIA published in this period present a high number of bibliographic reference errors, being the rate similar to other evaluated scientific periodical journals. This fact should be improved, and authors and reviewers should thoroughly check their references in order to avoid these mistakes, as they affect the quality standard of the publication.
Surprisingly few studies have evaluated the effects of technical editing rigorously. However there is some evidence that the 'package' of technical editing used by biomedical journals does improve papers. A substantial number of references in biomedical articles are cited or quoted inaccurately.