Searching multiple databases for systematic reviews: added value or diminishing returns? Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. Developing methods for systematic reviewing in health services delivery and organization: an example from a review of access to health care for people with learning disabilities. Part 1. Identifying the literature. Beyond Medline: reducing bias through extended systematic review search. Review of the usefulness of contacting other experts when conducting a literature search for systematic reviews. Searching one or two databases was insufficient for meta-analysis of observational studies. Publication bias and meta-analyses: a practical example. Lessons for search strategies from a systematic review, in The Cochrane Library, of nutritional supplementation trials in patients after hip fracture. Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? Reference accuracy in the dermatologic literature.
To explore whether searching specialised bibliographic databases identified additional relevant papers to those located by a Medline search for a systematic review of exercise therapy.
                Searches were performed in Medline, two further generalised medical databases (Embase, Cochrane Library) and four specialised databases (CancerLit, Cinahl, PsychInfo, SportDiscus) to identify controlled trials of exercise interventions for cancer patients.
                A total of 749 different publications were located through the search, of which 18 met inclusion criteria. Fifteen (83%) of these were identified through Medline and three (17%) from three individual specialised databases. A further seven studies meeting inclusion criteria were located through reference lists and contact with experts.
                In this example, searching Medline and additional specialised databases along with checking reference lists and contacting experts was the most effective means of ensuring that all relevant papers were included in the review. Searching Medline alone for systematic reviews of exercise or other unconventional therapies is likely to be inadequate. To describe where papers come from in a systematic review of complex evidence. Method Audit of how the 495 primary sources for the review were originally identified.
                Only 30% of sources were obtained from the protocol defined at the outset of the study (that is, from the database and hand searches). Fifty one per cent were identified by "snowballing" (such as pursuing references of references), and 24% by personal knowledge or personal contacts.
                Systematic reviews of complex evidence cannot rely solely on protocol-driven search strategies. Our objectives were to identify literature on: (i) theory, evidence and gaps in knowledge relating to the help-seeking behaviour of people with learning disabilities and their carers; (ii) barriers experienced by people with learning disabilities in securing access to the full range of health services; (iii) interventions which improve access to health services by people with learning disabilities.
                twenty-eight bibliographic databases, research registers, organizational websites or library catalogues; reference lists from identified studies; contact with experts; current awareness and contents alerting services in the area of learning disabilities.
                Inclusion criteria were English language literature from 1980 onwards, relating to people with learning disabilities of any age and all study designs. The main criteria for assessment was relevance to the Guilliford et al. model of access to health care (Gulliford et al. Access to health care. Report of a Scoping Exercise for the National Co-ordinating Centre for NHS Service Delivery and Organisation R & D (NCCSDO). London: NCCSDO, 2001), which was modified to the special needs of people with learning disabilities. Inclusion criteria focused on relevance to the model with initial criteria revised in light of literature identified and comments from a consultation exercise with people with learning disabilities, family and paid carers and experts in the field. Data abstraction was completed independently and selected studies were evaluated for scientific rigour and the results synthesized.
                In total, 2221 items were identified as potentially relevant and 82 studies fully evaluated.
                The process of identifying relevant literature was characterized by a process of clarifying the concept under investigation and sensitive search techniques which led to an initial over-identification of non-relevant records from database searches. Thesaurus terms were of limited value, forcing a reliance on using free-text terms and alternative methods of identifying literature to supplement and improve the recall of the database searches. A key enabler in identifying relevant literature was the depth and breadth of knowledge built up by the reviewers whilst engaged in this process. To evaluate the sensitivity and precision of various extended search methods in identifying randomized controlled trials (RCTs) for systematic reviews.
                Prospective analysis of extended search methods (specialized databases or trial registries, reference lists, hand-searching, personal communication, and Internet) used in two systematic reviews of RCTs. The gold standard was the total number of RCTs identified by major databases (MEDLINE, EMBASE, etc.) and extended search strategies combined. Sensitivity was the proportion of all known RCTs identified by any extended search method. Precision reflected the proportion of all items uncovered by any extended search method that actually were RCTs.
                The extended search identified 94 additional RCTs for the systematic reviews beyond those identified with the major databases. Specialized databases and trial registries had the highest sensitivity and precision for the lipid-lowering project (13.6% and 52.7%, respectively; p < .05) followed by scanning of reference lists (7.2% sensitivity and 41.9% precision; p <.05). Hand-searching was more effective than personal communication and Internet searching (1.7% sensitivity and 12.2% precision; p < .05). The acupuncture project had slightly different results, with the specialized databases and trial registries tied with the review of reference lists for highest sensitivity (14.2%). The precision followed the same trend as the lipid-lowering project (17.6% specialized databases; 8.3% reference lists; p < .05). A post-hoc analysis showed that 75 of the 94 RCTs were indexed in the major databases but missed by the major database search.
                Extended searching identified additional RCTs for the systematic reviews beyond those found in major databases. Specialized databases and trial registries were most effective. An important number of RCTs were missed by the major database search. Timing and accuracy of indexing may explain this finding. The definitive measure, whether there is an association between the method used to uncover RCTs, the quality of the items uncovered and their impact on systematic review results, is yet to be determined. nan To address methodologic issues in searching for observational studies by presenting database search methods and results.
                Results of two literature searches for publications reporting on observational studies of alcohol consumption and the risk of breast cancer and large bowel cancer were compared, to evaluate the sensitivity of various bibliographic databases and search strategies, including hand-searching reviews and meta-analyses.
                The target sensitivity of 90% of publications in the breast cancer search was achieved by starting with Medline, then adding Biosis, Embase, and SCI EXPANDED-SSCI, which provided a total of 72 (91%) of the 79 relevant publications. To reach a similar 89% sensitivity for large bowel cancer, at least Biosis, Dissertation Abstracts Online, Embase, ETOH, and Medline had to be searched, with the addition of hand search of reviews and meta-analyses.
                Limiting a search to one or two databases when conducting meta-analyses of observational studies will not provide a thorough summary of the existing literature. The findings support recommendations to implement a comprehensive search of electronic databases and the reference lists of recent review articles and meta-analyses. Publication bias is widely appreciated, but considerable time and effort are needed to locate and obtain data from unpublished randomized controlled trials (RCTs), those published in non-English language journals or those reported in the gray literature; for this publication, we will call this collection of trials the "gray+literature." However, excluding such trials from systematic reviews could introduce bias and give rise to misleading conclusions.
                We aimed to explore and quantify the impact of inclusion of gray+ literature on the results of all completed individual patient data (IPD) reviews coordinated by our group (13 meta-analyses). For each IPD review, results were calculated for RCTs fully published in English language journals and RCTs fully published in English language journals and the gray+literature.
                The IPD meta-analyses based only on RCTs that were fully published in English language journals tended to give more favorable results than those that included RCTs from the gray+literature. Although in most cases the addition of gray+data gave less encouraging results, moving the estimated treatment effect toward a null result, the direction of effect was not always predictable.
                We recommend that all systematic reviews should at least attempt to identify trials reported in the gray+literature and, where possible, obtain data from them. A key aim when conducting systematic reviews of randomized controlled trials (RCTs) is to include all of the evidence, if possible. Serious bias may result if trials are missed through inadequate search strategies.
                The objective was to evaluate the search plan for identifying RCTs in nutrition as part of a systematic review, in The Cochrane Library, of nutritional supplementation trials in patients after hip fracture.
                We identified potential studies by searching the electronic databases BIOSIS, CABNAR, CINAHL, EMBASE, HEALTHSTAR, and MEDLINE; reference lists in trial reports; and other relevant articles. We also contacted investigators and other experts for information and searched 4 nutrition journals by hand.
                We identified 15 RCTs that met the predefined inclusion criteria. The search plan identified 8 trials each in EMBASE, HEALTHSTAR, and MEDLINE and 7 in BIOSIS and CABNAR. BIOSIS was the only electronic database source of 2 trials. Eleven trials were identified by searching electronic databases and 2 unpublished trials were identified via experts in the field. We found one trial, published only as a conference abstract, by searching nutrition journals by hand. After publication of the protocol for the review in The Cochrane Library, we were informed of another unpublished trial.
                We found that a limited search plan based on only MEDLINE or one of the other commonly available databases would have failed to locate nearly one-half of the studies. To protect against bias, the search plan for a systematic review of nutritional interventions should be comprehensive. There is little guidance on how to identify useful evidence about the health effects of social interventions. The aim of this study was to assess the value of different ways of finding this type of information.
                Retrospective analysis of the sources of studies for one systematic review.
                Case study of a systematic review of the effectiveness of interventions in promoting a population shift from using cars towards walking and cycling.
                Only four of the 69 relevant studies were found in a "first-line" health database such as Medline. About half of all relevant studies were found through the specialist Transport database. Nine relevant studies were found through purposive internet searches and seven relevant studies were found by chance. The unique contribution of experts was not to identify additional studies, but to provide more information about those already found in the literature.
                Most of the evidence needed for this review was not found in studies indexed in familiar literature databases. Applying a sensitive search strategy across multiple databases and interfaces is very labour intensive. Retrospective analysis suggests that a more efficient method might have been to search a few key resources, then to ask authors and experts directly for the most robust reports of studies identified. However, internet publications and serendipitous discoveries did make a significant contribution to the total set of relevant evidence. Undertaking a comprehensive search may provide unique evidence and insights that would not be obtained using a more focused search. The reference list is an important part of a scientific article. To be useful it must be accurate.
                The purpose of this study was to evaluate the accuracy of references in the dermatologic literature.
                We randomly selected 240 references (60 per journal) from the Archives of Dermatology, the British Journal of Dermatology, this Journal, and the Journal of Investigative Dermatology and checked them against the original articles.
                The overall rate of citation error (the information identifying the source) was 41%, and the quotation error (inconsistency between the statement referenced and the original source) was 35%. Only 36% of references were free of error.
                This study shows that the rate of citation and quotation errors is unacceptably high in the dermatologic literature, which significantly diminishes the value of the reference list.
There is some evidence to support the use of checking reference lists for locating studies in systematic reviews. However, this evidence is derived from weak study designs. In situations where the identification of all relevant studies through handsearching and database searching is difficult, it would seem prudent that authors of reviews check reference lists to supplement their searching. The challenge, therefore, is for review authors to recognize those situations.