Abstract
Objectives: Various minimal clinically important difference (MCID) threshold estimation techniques have been applied to seasonal allergic rhinitis (SAR). The objectives of this study are to (i) assess the difference in magnitude of alternative SAR MCID threshold estimates and (ii) evaluate the impact of alternative MCID estimates on health technology assessment (HTA). Methods: Data describing change from baseline of the reflective Total Nasal Symptom Score (rTNSS) for four intranasal SAR treatments were obtained from United States Food and Drug Administration-approved prescribing information. Treatment effects were then compared with anchor-based MCID thresholds derived by Barnes et al. and thresholds obtained from an Agency for Healthcare Research and Quality (AHRQ) panel.
Abstract
The change in rTNSS score from baseline, represented as the average of the twice-daily recorded scores of the rTNSS, was -2.1 (p < .001) for azelastine hydrochloride 0.10%, 1.35 (p = .014) for ciclesonide, and -1.47 (p < .001) for fluticasone furoate. The change in the rTNSS score from baseline, represented by sum of the AM and PM score, was -2.7 for MP-AzeFlu (p < .001). The rTNSS change from baseline for each product was compared with anchor-based MCID threshold and the AHRQ panel estimates. Comparison of the observed treatment effect to the anchor-based and AHRQ panel MCID thresholds results in different conclusions, with clinically important differences being inferred when anchor-based estimates serve as the reference point. Conclusion: The AHRQ panel MCID threshold for the rTNSS was twelve times larger than the anchor-based estimates resulting in conflicting recommendations on whether different SAR treatments provide clinically meaningful benefit.

In 1993, Kozma and colleagues advanced a new framework for outcomes research known as the "ECHO" model (1) . Using this new framework, the authors cautioned the medical community against relying predominantly on efficacy data generated from clinical trial programs. Rather, the authors suggested that medical decision making should simultaneously consider the economic, clinical, and humanistic impacts of alternative treatment options.

During the same time period, significant advancements were being made in humanistic outcomes research. Due to the contributions of interdisciplinary teams around the world, a proliferation of studies designed to develop and validate patient reported outcome (PRO) measures occurred. These scientifically validated measures provided the conduit for gathering general or disease related patient reports on various aspects of either their disease or treatment. With the availability of patient reported outcomes data, thresholds quantifying a clinically meaningful improvement could be developed. These thresholds are now known as minimal clinically important difference (MCID) thresholds.
Alternative Methods to Derive MCID Thresholds
In recent years, the analytic rigor required to support MCID threshold estimation has increased. The work of numerous interdisciplinary teams including biostatisticians, epidemiologists, physicians, and others have culminated in the identification of three different and unique strategies for MCID development (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) (13) . Table 1 provides a summary of the various techniques to derive MCID threshold estimates and provides examples of them.
Alternative Methods to Derive MCID Thresholds
Two quantitative strategies, the distribution-based and anchor-based MCID threshold estimation methods, have been advanced more recently in the scientific literature. (14) 0.59 units Assesses the measurement precision of a given PRO measure
Alternative Methods to Derive MCID Thresholds
Sample-dependent and difficult to ascertain the clinical relevance for a given change Anchor-based MCID estimates (14) 0.23-0.28 units Uses validated PRO data as the anchor for the estimate and is not sample-dependent
Alternative Methods to Derive MCID Thresholds
Computationally more involved MCID, minimum clinically important difference; PRO, patient reported outcome.
Alternative Methods to Derive MCID Thresholds
Distribution-based MCID estimates report the change in a PRO measure based on the normal variability observed in the data and, in general, assume that a change greater than X times the baseline standard deviation is clinically meaningful (14) . The utility of distribution-based methods may be limited because they are based on statistical reasoning only. Also, variations in standard deviations may be observed when PRO data are obtained from heterogeneous patient populations. Because of these limitations, anchor-based threshold estimates are generally perceived as the more rigorous of the two techniques (14) . The third strategy involves the development of MCID threshold estimates by technical expert panelists. Generally this method uses clinical experience to derive the estimates instead of patient reported outcome data. When all three types of threshold estimates are available, thresholds developed based upon clinical expert opinions have been recommended as a supplementary strategy to existing anchor-and distribution-based estimates of MCID (14) .
MCID Estimates and Technology Assessment of SAR Treatment Options: A Case Study
The Total Nasal Symptom Score (TNSS) is a PRO measure required by marketing authorization authorities (eg, the United States Food and Drug Administration) to assess patient perceptions of the benefits of alternative treatments for seasonal allergic rhinitis (SAR) (15) . The reflective Total Nasal Symptom Score (rTNSS) measures the overall effectiveness of a treatment in controlling symptoms of a pre-specified (e.g., 12 hr) period of time, and AM and PM reports are typically provided by patients. Patients report their symptom severity on a scale of zero (no symptoms) to three (severe symptoms) for up to four different symptom types. The four symptom types assessed by the rTNSS include rhinorrhea, nasal congestion, nasal itching, and sneezing. Computation of treatment effectiveness using the TNSS as the primary endpoint for clinical trials for SAR treatments can be derived through various scoring mechanisms. In general, however, treatment effectiveness is defined as the change in TNSS score from baseline. With respect to calculating the score at any time point in the study, one technique uses an average of the score recorded by the patient in the morning and the evening. Another method of scoring involves summing of the morning and evening patient reported data.
MCID Estimates and Technology Assessment of SAR Treatment Options: A Case Study
Using patient reported data derived from the TNSS, Barnes and colleagues derived MCID estimates using several analytical techniques. A more in-depth discussion of data sources and statistical methods used by Barnes et al. to generate MCID estimates using data collected from patients with SAR is described elsewhere (14) . However, one of the MCID thresholds derived used the preferred direct anchor-based approach with estimated MCIDs ranging from 0.28 units (95% confidence interval [CI]: -0.18-0.73) and 0.23 units (95% CI: -0.16-0.62). This estimate is used to determine whether four different intranasal steroids' efficacy data as found in the approved prescribing information would likely be of a magnitude that patients would perceive the improvements to be clinically meaningful. Table 2 provides a summary of the data that is used to complete the comparison. The focus of the data presented is solely on current representatives of intranasal steroid and antihistamine treatments. A wider complementary analysis representing further treatment options for SAR is described elsewhere (16) . When the absolute value of treatment effect is larger than the absolute value of the estimated MCID thresholds the treatment is assumed to not only provide a statistically significant improvement in patient's symptoms but also a clinically meaningful improvement as well. Data provided in Table 2 illustrate that all treatments advance both a statistically significant and clinically meaningful improvement in symptom relief for SAR patients.
MCID Estimates and Technology Assessment of SAR Treatment Options: A Case Study
In July 2013, the Agency for Healthcare Research and Quality (AHRQ) released a comparative effectiveness review of SAR treatments (17) . Despite the availability of anchor-based estimates in 2010 (14) , the AHRQ reviewers used consensus-based technical panel MCID threshold estimates (17) . Two panel members considered a 4-point change While responsiveness and MCID threshold estimation may be related, the two concepts should not be inferred as being interchangeable as one focuses on individual patient effects and the other compares population means. Irrespective of the underlying rationale for the selection, a 30% (3.6 units) MCID threshold derived from the AHRQ panel substantially exceeds the anchor-based MCID threshold derived through rigorous statistical testing of patient reported rTNSS data. Application of the 3.6 threshold in a manner consistent with the previous evaluation would suggest that none of the evaluated intranasal products provide clinically meaningful benefits in patient populations for which they have been approved and routinely used in clinical practice.
DISCUSSION
Healthcare decision makers are faced with a difficult task on a daily basis: assessing the quality of evidence for multiple therapies and interpreting the plethora of data to guide reimbursement and coverage determinations that affect patients' lives, physically and monetarily. Within the United States (US), payers may rely upon data provided within Academy of Managed Care Pharmacy Format for Formulary Submissions (19) and supplemental systematic reviews and additional reports. External to the United States, HTA organizations such as the National Institute for Health and Care Excellence in the United Kingdom and the Institute for Quality and Efficiency in Health Care in Germany provide guidance about whether the value of an individual product may warrant reimbursement (20;21) .
DISCUSSION
Over the past few decades, researchers have sought to obtain high quality patient reported data and quantify MCID thresholds that can be used to provide clinicians and reimbursement authorities with a secondary reference threshold to evaluate the clinical effectiveness of alternative therapies (12) . The MCID, first defined by Jaeschke et al., is "the smallest difference in score in the domain of interest which patients would perceive as beneficial and which would mandate, in the absence of troublesome side effects and excessive cost, a change in the patient's management" (3). Thus, MCIDs may be an important component of medical decision making if populations of patients are to maintain sufficient access to treatment options most likely to provide them with meaningful improvement in symptoms.
DISCUSSION
Furthermore, the case study above demonstrated that reimbursement decisions may vary based upon the MCID threshold estimation technique used. To mitigate the risk of denying patient access to treatment options that may result in clinically important improvements in their treatment, all available methods should be considered and presented in health technology assessment reports. A critical assessment should discuss differences in obtained results and check whether the applied MCID is sensitive for detecting relevant effects.
DISCUSSION
Before clinicians can routinely use MCID to guide decisions specific to treatment plans of individual patients, much work is needed by the research community in collaboration with payers and reimbursement authorities. There may be merit in bringing experts from around the globe together to discuss many of the issues identified in this study. For example, this study identified that the TNSS has favorable endorsement for marketing authorization and regulatory bodies for use in pivotal clinical trials. Once a PRO gains this favorable endorsement, should there be an immediate focus on developing MCID threshold estimates in a uniform manner to guide subsequent reimbursement/payer decisions? Also, what entity, if any, should assume responsibility for monitoring ranges of MCID threshold estimates and commissioning action when large variations exist? Finally, what group should be identified as monitoring the medical literature and designated terms associated with MCID threshold estimation? Encouraging standardization around such terminology may support ease of access of such data when it is needed by clinicians or reimbursement authorities.
DISCUSSION
As with any evaluation, this review and associated case study have certain limitations. First, the statistical analysis supporting the derivation of MCIDs using alternative methods is simply reviewed. Additional detail is provided for the review in the study by Barnes et al. (14) . In addition, the therapeutic alternatives in the evaluation are not complete and only a few intranasal treatment options for SAR are considered. The authors make no apology for the simplicity. Quality, transparency and simplicity in decision making should always be a desired goal in medicine. The narrow focus on a select few current treatment options for SAR was deemed adequate to illustrate the methodological points and was motivated by the authors' awareness that additional research was ongoing to support further therapeutic options for the treatment of SAR (16) .
CONCLUSION
Medical decision making today is greatly enhanced because of high quality outcomes research. Clinical trials continue around the global and provide healthcare decision makers insights on the potential advantages or disadvantages of alternative treatment options for a given disease state. Reliance, however, on this information solely for the purposes of medical decision making continues to be an area of public debate.
CONCLUSION
With the proliferation of high quality humanistic outcomes research data, medical decision makers may now have at their disposal high quality evidence to better understand the influence of alternative technologies on patient reported health status. The translation of PRO data into methodologically sound MCID estimates and the consistent application of rigorously developed MCID threshold estimates are important steps in attaining the appropriate balance between clinical and humanistic outcomes. Individuals and entities conducting health technology assessment have a professional and ethical responsibility to use MCID estimates derived from rigorous statistical analysis of population based patient reported outcomes when available.
CONFLICTS OF INTEREST
This study was funded by Meda Pharmaceutical. Dr. Munzel was employed by Meda Pharma GmbH when the study was conducted and has no conflict of interest with regard to this study. Dr.'s Carroll and Morland are employees of Xcenda, a contractor to Meda at the time of study completion. Dr. Brixner received personal fees from Xcenda during the conduct of this study. She also reports receipt of personal funds and/or grants from Millcreek Outcomes Group, Abbott, Novo Nordisk, and Certara outside of the submitted work. Dr. Meltzer received personal fees from Meda during the conduct of this study. In addition, Dr. Meltzer reports personal fees and grants from Allergan, AstraZeneca, Boehringer Ingelheim, Circassia, GlaxoSmithKline, Greer, Johnson & Johnson, Merck, Mylan, Regeneron/Sanofi, Sunovion, 3E Therapeutics, Takeda, Teva, and Valeant outside the conduct of the submitted work. Dr. Lipworth reports grants and personal fees from Meda during the conduct of this study, as well as the receipt of personal fees, grants, and/or non-financial support from Dr. Reddy, Cipla, Novartis, Boehringer-Ingelheim, Teva, Astra Zeneca, Janssen, Roche, and Chiesi outside the conduct of the submitted work.

{'paper_id': '18844088', 'title': 'IMPLICATION OF ALTERNATIVE MINIMAL CLINICALLY IMPORTANT DIFFERENCE THRESHOLD ESTIMATION METHODS ON TECHNOLOGY ASSESSMENT', 's2_url': 'https://api.semanticscholar.org/CorpusID:18844088', 'source': 'S2ORC'}