Preparing and updating systematic reviews of randomized controlled trials of health care. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Review of the usefulness of contacting other experts when conducting a literature search for systematic reviews. Searching multiple databases for systematic reviews: added value or diminishing returns? Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? Strategies for improving the quality of health care in maternal and child health in low- and middle-income countries: an overview of systematic reviews. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. Identifying null meta-analyses that are ripe for updating. Methodologists and their methods. Do methodologists write up their conference presentations or is it just 15 minutes of fame?
People providing health care tend to turn to review articles rather than reports of primary research when seeking information to guide their practices. For this reason, the process of reviewing the results of primary research must respect scientific principles. Reviews must be kept up to date and disseminated in appropriate ways to the people who make decisions about health care, including policy makers, practitioners, and users of the health services. This article describes the methods developed and used in an attempt to address these challenges for care during pregnancy and childbirth. nan nan To explore whether searching specialised bibliographic databases identified additional relevant papers to those located by a Medline search for a systematic review of exercise therapy.
                Searches were performed in Medline, two further generalised medical databases (Embase, Cochrane Library) and four specialised databases (CancerLit, Cinahl, PsychInfo, SportDiscus) to identify controlled trials of exercise interventions for cancer patients.
                A total of 749 different publications were located through the search, of which 18 met inclusion criteria. Fifteen (83%) of these were identified through Medline and three (17%) from three individual specialised databases. A further seven studies meeting inclusion criteria were located through reference lists and contact with experts.
                In this example, searching Medline and additional specialised databases along with checking reference lists and contacting experts was the most effective means of ensuring that all relevant papers were included in the review. Searching Medline alone for systematic reviews of exercise or other unconventional therapies is likely to be inadequate. There is little guidance on how to identify useful evidence about the health effects of social interventions. The aim of this study was to assess the value of different ways of finding this type of information.
                Retrospective analysis of the sources of studies for one systematic review.
                Case study of a systematic review of the effectiveness of interventions in promoting a population shift from using cars towards walking and cycling.
                Only four of the 69 relevant studies were found in a "first-line" health database such as Medline. About half of all relevant studies were found through the specialist Transport database. Nine relevant studies were found through purposive internet searches and seven relevant studies were found by chance. The unique contribution of experts was not to identify additional studies, but to provide more information about those already found in the literature.
                Most of the evidence needed for this review was not found in studies indexed in familiar literature databases. Applying a sensitive search strategy across multiple databases and interfaces is very labour intensive. Retrospective analysis suggests that a more efficient method might have been to search a few key resources, then to ask authors and experts directly for the most robust reports of studies identified. However, internet publications and serendipitous discoveries did make a significant contribution to the total set of relevant evidence. Undertaking a comprehensive search may provide unique evidence and insights that would not be obtained using a more focused search. There are many systematic reviews of continuing education programmes and educational strategies for quality improvement in health care. Most of the reviewed studies are one-off evaluations rather than impact evaluations with long-term follow-up. There are few systematic reviews of organisational, financial and regulatory interventions, and few high-quality studies. These interventions are probably as or more important than educational strategies, although they are less well evaluated. Few studies have been undertaken in low- and middle-income countries (LMIC) or that address maternal and child health (MCH). Thus, the results of the available studies and reviews need to be interpreted cautiously when applied to LMIC. Interactive workshops, reminders and multifaceted interventions can improve professional practice, and they generally have moderate effects. Educational outreach visits consistently improve prescribing but have variable effects on other behaviours. Audit and feedback interventions have variable effects on professional practice, but most often these are small to moderate effects. Mass-media and patient-mediated interventions may change professional practice. Multifaceted interventions that combine several quality-improvement strategies are also effective but may not be more so than single interventions. While all of these strategies are applicable to MCH in LMIC, the applicability of the results to rural settings, in particular, may be limited. Use of these strategies could exacerbate inequalities, and this should be taken into consideration when planning implementation. Scaling up and sustainability may be difficult to achieve in LMIC contexts and need careful consideration. The use of financial interventions has not been well studied; financial incentives and disincentives may be difficult to use effectively and efficiently, although their impact on practice needs to be considered. Organisational interventions are likely to be important, given that there are often underlying organisational or system problems. Regulatory interventions have not been well evaluated, but may sometimes be both inexpensive and effective. There are no 'magic bullets' or simple solutions for ensuring the quality of health care services. Interventions should be selected or tailored to address the underlying reasons for a failure to deliver effective services. Decision-makers should select the most appropriate interventions for specific problems. This requires a governance structure that clearly assigns responsibility for quality-improvement activities, priority setting, selection and design of interventions, and evaluation. To describe where papers come from in a systematic review of complex evidence. Method Audit of how the 495 primary sources for the review were originally identified.
                Only 30% of sources were obtained from the protocol defined at the outset of the study (that is, from the database and hand searches). Fifty one per cent were identified by "snowballing" (such as pursuing references of references), and 24% by personal knowledge or personal contacts.
                Systematic reviews of complex evidence cannot rely solely on protocol-driven search strategies. As an increasingly large number of meta-analyses are published, quantitative methods are needed to help clinicians and systematic review teams determine when meta-analyses are not up to date.
                We propose new methods for determining when non-significant meta-analytic results might be overturned, based on a prediction of the number of participants required in new studies. To guide decision making, we introduce the "new participant ratio", the ratio of the actual number of participants in new studies to the predicted number required to obtain statistical significance. A simulation study was conducted to study the performance of our methods and a real meta-analysis provides further evidence.
                In our three simulation configurations, our diagnostic test for determining whether a meta-analysis is out of date had sensitivity of 55%, 62%, and 49% with corresponding specificity of 85%, 80%, and 90% respectively.
                Simulations suggest that our methods are able to detect out-of-date meta-analyses. These quick and approximate methods show promise for use by systematic review teams to help decide whether to commit the considerable resources required to update a meta-analysis. Further investigation and evaluation of the methods is required before they can be recommended for general use. To assess the extent to which abstracts of methodological research, initially presented at meetings on systematic reviews, have gone on to be published as full articles.
                Full publication was assessed in three ways: a search was carried out of The Cochrane Library; a search was conducted using MEDLINE; and a questionnaire was sent to the contact author of each abstract.
                Approximately half of the abstracts had not been, or were unlikely ever to be, published in full.
                The rate of full publication of abstracts related to the methodology of systematic reviews seems similar to that for randomized trials.
Little research has been conducted on when and how to update systematic reviews and the feasibility and efficiency of the identified approaches is uncertain. These shortcomings should be addressed in future research.