The meta-analyses by Gillies and colleagues were based on adverse events occurring in randomized controlled trials that compared amoxicillin (as monotherapy or in combination with clavulanic acid) against placebo.1 Intriguingly, amoxicillin monotherapy was not associated with significant harm (such as diarrhea, rash or nausea), whereas amoxicillin–clavulanic acid was associated with a significant increase in the risk of diarrhea. Gillies and colleagues judged their meta-analyses to have “low risk of bias,” which perhaps gives us greater reassurance about the apparent lack of harm with amoxicillin.
Nonetheless, several questions come to mind. How confident should we be in accepting these “null” findings, and can clinicians continue to prescribe this antibiotic without worrying too much about unintended consequences? Can we trust that there is a genuinely low risk of bias (i.e., high internal validity) in the reported absence of harm? Might there be reporting biases that prevent us from seeing the whole picture? Finally, can perceived safety based on the experience of participants in randomized trials be reliably extrapolated to patients in real-life clinical practice?
In practice, prescribing decisions are complicated by the risk of adverse effects and by emerging bacterial resistance from misuse (or overuse) of antibiotics.3 In a recent survey study, 55% of family physicians in the United Kingdom felt under pressure (from patients) to prescribe antibiotics.4 It is important that decisions to prescribe antibiotics be informed by high-quality safety data from systematic reviews.
We must recognize the inherent difficulties of accurately capturing rare or unexpected adverse events that were not specified or defined beforehand.5 Only 18% of the trials in the study by Gillies and colleagues used a diary to record harms,1 but such a lack of rigorous ascertainment of harm leads to “non-differential misclassification,” with bias toward the null (i.e., lower estimates of risk of harm).6 To put it simply, if you don’t look properly, you won’t find — a combination of poor monitoring with a lack of clear case definitions could lead to misclassification or nondetection of genuine adverse reactions.5 The paucity of recorded events in the trials included in the analysis by Gillies and colleagues potentially creates an illusion of safety. However, we wonder if the greater perceived risk of diarrhea in trials involving amoxicillin–clavulanic acid is related to more rigorous ascertainment in those trials, with null findings for amoxicillin monotherapy simply reflecting poorer ascertainment of adverse effects rather than a genuine safety advantage.
Potential bias from selective outcome reporting may also reduce confidence in pooled estimates of harm. For example, adverse events might have been recorded, but researchers did not report the outcomes. The exact reasons for selective non-reporting are unclear, but certain situations may arise to create a positive impression. For instance, perhaps researchers selectively omitted reports of harm to generate a favourable benefit–harm profile, or because they felt that adverse effects were unimportant. Equally, an excess of adverse events with antibiotic (relative to placebo) might have gone unreported if the statistical analysis did not yield significant results.
In the 45 primary studies that Gillies and colleagues included in their systematic review, harms follow-up was no more than one week in nine of the trials, and 17 trials did not specify the duration of harms follow-up.1 The generalizability of findings is questionable when highly selected trial participants are followed for only a short time.5 If antibiotic-associated diarrhea is a late adverse event, short-term studies may fail to record any harm (type 2 error). Nonrandomized community studies may therefore generate higher estimates of harm if participants had more comorbidities, with longer duration of antibiotic use and longer follow-up.
Poor monitoring, lack of clear case definitions and missing data mean that genuine adverse reactions may go undetected or be misclassified.
Systematic reviews of harm should explicitly assess this risk of bias toward the null to prevent a false sense of security (type 2 error), whereby a drug is erroneously declared safe or not significantly different from the comparator.
Quality assessment tools for adverse events allow findings of “no significant harm” to be subjected to the same rigorous scrutiny as claims of efficacy.
Absence of evidence of harm should not be construed as evidence of absence of harm, and there is a pressing need for unrestricted access to complete trial datasets for both beneficial and adverse effects.
Contributors: Both authors contributed substantially to conception and design of the article and interpretation of the data; drafted the article; gave final approval of the version to be published; and agree to act as guarantors of the work in ensuring that questions related to any part of the work are appropriately investigated and resolved.
. Common harms from amoxicillin: a systematic review and meta-analysis of randomized placebo-controlled trials for any indication. CMAJ 2014; Nov. 17 [Epub ahead of print].
. Lack of clarity in reports of adverse events: Is there any harm? Pain 2013;154:183–4.
. A systematic review and meta-analysis of the effects of antibiotic consumption on antibiotic resistance. BMC Infect Dis 2014;14:13.
. GPs feel pressurised to prescribe unnecessary antibiotics, survey finds. BMJ 2014;349:g5238.
. Comprehensive evaluations of the adverse effects of drugs: importance of appropriate study selection and data sources. Ther Adv Drug Saf 2011;2:59–68.
. Bias. J Epidemiol Community Health 2004;58:635–41.
. AHRQ series paper 4: assessing harms when comparing medical interventions: AHRQ and the effective health-care program. J Clin Epidemiol 2010;63:502–12.
. Dexamethasone and haemorrhage risk in paediatric tonsillectomy: a systematic review and meta-analysis. Br J Anaesth 2014;113:23–42.
. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol 2011; 64:401–6.
. Increasing value and reducing waste: addressing inaccessible research. Lancet 2014;383: 257–66.