Abstract
Background: iPrevent is an online breast cancer (BC) risk management decision support tool. It uses an internal switching algorithm, based on a woman's risk factor data, to estimate her absolute BC risk using either the International Breast Cancer Intervention Study (IBIS) version 7.02, or Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm version 3 models, and then provides tailored risk management information. This study assessed the accuracy of the 10-year risk estimates using prospective data. Methods: iPrevent-assigned 10-year invasive BC risk was calculated for 15 732 women aged 20-70 years and without BC at recruitment to the Prospective Family Study Cohort. Calibration, the ratio of the expected (E) number of BCs to the observed (O) number and discriminatory accuracy were assessed. Results: During the 10 years of follow-up, 619 women (3.9%) developed BC compared with 702 expected (E/O ¼ 1.13; 95% confidence interval [CI] ¼1.05 to 1.23). For women younger than 50 years, 50 years and older, and BRCA1/2-mutation carriers and noncarriers, E/O was 1.04 (95% CI ¼ 0.93 to 1.16), 1.24 (95% CI ¼ 1.11 to 1.39), 1.13 (95% CI ¼ 0.96 to 1.34), and 1.13 (95% CI ¼ 1.04 to 1.24), respectively. The C-statistic was 0.70 (95% CI ¼ 0.68 to 0.73) overall and 0.74 (95% CI ¼ 0.71 to 0.77), 0.63 (95% CI ¼ 0.59 to 0.66), 0.59 (95% CI ¼ 0.53 to 0.64), and 0.65 (95% CI ¼ 0.63 to 0.68), respectively, for the subgroups above. Applying the newer IBIS version 8.0b in the iPrevent switching algorithm improved calibration overall (E/O ¼ 1.06, 95% CI ¼ 0.98 to 1.15) and in all subgroups, without changing discriminatory accuracy. Conclusions: For 10-year BC risk, iPrevent had good discriminatory accuracy overall and was well calibrated for women aged younger than 50 years. Calibration may be improved in the future by incorporating IBIS version 8.0b.

Background: iPrevent is an online breast cancer (BC) risk management decision support tool. It uses an internal switching algorithm, based on a woman's risk factor data, to estimate her absolute BC risk using either the International Breast Cancer Intervention Study (IBIS) version 7.02, or Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm version 3 models, and then provides tailored risk management information. This study assessed the accuracy of the 10-year risk estimates using prospective data. Methods: iPrevent-assigned 10-year invasive BC risk was calculated for 15 732 women aged 20-70 years and without BC at recruitment to the Prospective Family Study Cohort. Calibration, the ratio of the expected (E) number of BCs to the observed (O) number and discriminatory accuracy were assessed. Results: During the 10 years of follow-up, 619 women (3.9%) developed BC compared with 702 expected (E/O ¼ 1.13; 95% confidence interval [CI] ¼1.05 to 1.23). For women younger than 50 years, 50 years and older, and BRCA1/2-mutation carriers and noncarriers, E/O was 1.04 (95% CI ¼ 0.93 to 1.16), 1.24 (95% CI ¼ 1.11 to 1.39), 1.13 (95% CI ¼ 0.96 to 1.34), and 1.13 (95% CI ¼ 1.04 to 1.24), respectively. The C-statistic was 0.70 (95% CI ¼ 0.68 to 0.73) overall and 0.74 (95% CI ¼ 0.71 to 0.77), 0.63 (95% CI ¼ 0.59 to 0.66), 0.59 (95% CI ¼ 0.53 to 0.64), and 0.65 (95% CI ¼ 0.63 to 0.68), respectively, for the subgroups above. Applying the newer IBIS version 8.0b in the iPrevent switching algorithm improved calibration overall (E/O ¼ 1.06, 95% CI ¼ 0.98 to 1.15) and in all subgroups, without changing discriminatory accuracy. Conclusions: For 10-year BC risk, iPrevent had good discriminatory accuracy overall and was well calibrated for women aged younger than 50 years. Calibration may be improved in the future by incorporating IBIS version 8.0b.

Clinical decision support tools can help integrate and personalize evidence to assist clinicians and patients to make betterinformed decisions (1) . Tools that are computerized, personalized, and accessible not only to clinicians but also to patients themselves have better uptake and result in greater adherence to recommended practice (2) . iPrevent is a new, evidence-based, personalized breast cancer (BC) risk assessment and risk management decision support tool (3) developed to assess risk across the entire risk spectrum. It is freely available online at https://www.petermac.org/iprevent. It is designed for collaborative use between women and their clinicians and was developed to be consistent with best practice in risk communication. iPrevent assesses a woman's personal BC risk and provides tailored risk management advice, including information about chemoprevention, risk-reducing surgeries, lifestyle modification, and screening ( Figure 1 ) (4) .

There are many available BC-risk prediction models, but their use in the clinic can be hampered by a lack of clear guidelines on which models should be applied in which circumstances. iPrevent chooses from two commonly used pedigree-based models, according to the algorithm shown in Figure 2- (7) are well calibrated and outperform other commonly used risk models like Gail (8) and BRCAPRO (9) , are applicable to all women across the spectrum of absolute familial risk. In addition, IBIS incorporates nonfamilial risk factors and is appropriate for women without a BC family history. Other commonly used BC models such as the Gail (8) or Breast Cancer Surveillance Consortium model (10) have been developed and validated only for women at average risk.

After providing a personal BC risk estimate, iPrevent gives a menu of appropriate potential risk management interventions based on Australian National Guidelines (11, 12) , applying published relative risk reductions for relevant interventions (eg, 33% relative risk reduction for 5 years of tamoxifen use) (13) to give personalized, detailed, quantitative information on how the intervention changes a woman's absolute BC risk. Women and their doctors can choose to see this information presented as statistics, pictograms, and/or graphs. Individualized lifestyle advice is also provided, based on the risk factor data the woman supplies.

A pilot study of iPrevent use by women and their clinicians found it had good usability and acceptability and suggested that it might improve the accuracy of BC risk perception and enhance BC prevention knowledge without increasing BC worry or anxiety (14) .

The primary aim of this study was to assess the calibration and discrimination of the 10-year BC risk estimates provided by the current online version of iPrevent, using data from a prospective cohort. During the course of this study, v8.0b of IBIS was released, which updated model parameters for the association between hormone replacement therapy (HRT) and BC risk and also allowed for optional inclusion of mammographic density and single nucleotide polymorphism (SNP) data. Hence, we also prospectively validated the iPrevent switching algorithm, incorporating IBIS v8.0b.
Methods
This validation study was performed using data from the Prospective Family Study Cohort (15) , which pooled data from two cohorts: the Breast Cancer Family Registry (BCFR) (16) and the Kathleen Cuningham Foundation Consortium for Research into Familial Breast Cancer (kConFab) Follow-Up Study (17) . These cohorts are enriched for women with an increased familial risk of BC. The BCFR and kConFab used the same risk factor questionnaire at enrollment, which collected data on demographics, height, weight, breast and ovarian surgeries, reproductive history, and lifestyle factors. The cancer family history style and medical and family history and, using that information, provides 10-year breast cancer risk estimates as well as lifetime risk estimates (not shown). B) iPrevent provides a menu of appropriate risk management options for each woman depending on her category of risk, based on her absolute risk estimate and Cancer Australia guidelines. An example for a woman at moderate risk of breast cancer. C) iPrevent provides personalized estimates of the absolute reduction in breast cancer risk that may be expected with prevention strategies such as tamoxifen.
Methods
questions asked about breast and other cancers diagnosed in participants and their first-and second-degree relatives, including ages at diagnosis and age last known to be alive, for all family members. Screening for germline BRCA1 and BRCA2 mutations by the BCFR and kConFab typically involved testing the affected proband and/or the youngest affected woman in each family, followed by cascade testing of other family members for the family-specific mutation if present in the proband. Incident BC diagnoses were verified through pathology reports for 84% of cases. Participants provided written informed consent and were followed for a median of 10 years. The cohort protocols were approved by institutional review boards at each of the respective institutions.
Methods
At baseline, there were 18 856 women in the Prospective Family Study Cohort without a personal history of invasive or in situ BC (15) . We excluded from the analyses those women who had bilateral mastectomy prior to cohort entry (n ¼ 113), those with follow-up time of less than 2 months (n ¼ 517), those with prior history of ovarian cancer (n ¼ 316), those younger than 20 years or older than 70 years at baseline (n ¼ 2082), and those for whom inadequate pedigree data were available (n ¼ 96), leaving 15 732 eligible participants from 6694 families.
Methods
We manually applied the iPrevent switching algorithm to the entire cohort (which allowed us to consider the impact of two versions of IBIS: 7.02 and 8.0b). We first determined for each participant whether iPrevent would use the IBIS or BOADICEA model to generate the iPrevent risk estimate (Figure 2 ). The software packages IBIS v7.02 (http://www.ems-trials.org/riskevaluator/) and BOADICEA v3 (https://pluto.srl.cam.ac.uk/cgi-bin/bd3/ v3/bd.cgi) were then used to assign 10-year BC risks. These risk estimates were then further modified based on preventive measures used prior to baseline by applying a 33% reduction in relative risk for women who had taken tamoxifen for BC prevention prior to baseline and a 50% reduction in relative risk for women who had had a bilateral oophorectomy before age 45 years (although the validity of the latter has recently been questioned [18] , and this may need future modification). The same process was subsequently undertaken using IBIS v8.0b, but mammographic density and SNP-based polygenic risk (which can be entered in IBIS v8.0b but not v7.2) were not available for this dataset (19). BRCA2-mutation status, the iPrevent 10-year breast cancer risk estimate was derived from the IBIS model for 9749 participants and the BOADICEA model for the remaining 5983 participants. # IBIS-or BOADICEA-derived risk estimates were further modified to obtain the iPrevent risk estimate by applying a 33% relative risk reduction for women who had taken tamoxifen for breast cancer prevention prior to baseline and a 50% relative risk reduction for women who had had a bilateral oophorectomy before age 45 years. AH ¼ atypical hyperplasia; BOADICEA ¼ Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm; BRCA mutation ¼ BRCA1 or BRCA2 pathogenic germline mutation; IBIS ¼ International Breast Cancer Intervention Study; LCIS ¼ lobular carcinoma in situ.
Statistical Analysis
The calibration and discriminatory accuracy of the 10-year iPrevent BC risk estimates were assessed for the overall cohort and also for subgroups 1) younger than 50 years at baseline, 2) 50 years or older at baseline, 3) BRCA1-and BRCA2-mutation carriers (combined), and 4) noncarriers. Follow-up time was censored at date of in situ BC diagnosis, bilateral mastectomy, death, loss to follow-up, or on reaching 10 years. If follow-up time was less than 10 years, we used the method described by Amir and colleagues (20) to adjust the predicted risk to the available follow-up time.
Statistical Analysis
For calibration, the expected number of invasive BC cases based on the iPrevent-assigned risk (E) was compared with the observed number of cases (O) (21) . The 95% confidence intervals (CI) for the ratio of expected to observed numbers (E/O) was calculated by E/O*EXP(6 1.96*SQRT[1/O]) (22) . For the overall sample and in the younger than age 50 years, 50 years and older, and noncarrier subgroups, the mean iPrevent-assigned risk to observed BC incidence was calculated overall and for each decile of risk, with separate deciles used for BRCA1-and BRCA2-mutation carriers. To compare calibration between IBIS v7.02 and v8.0b, we calculated the estimated calibration index (ECI) defined as the average squared difference between observed and predicted risk. Values of ECI closer to zero indicate better calibration (23) .
Statistical Analysis
To evaluate discriminatory accuracy, the receiver operating characteristic curves for the development of invasive BC within 10 years, overall, and for the subgroups defined above were plotted and the C-statistic computed. A C-statistic of 0.6-0.7 is considered "sufficient," 0.7-0.8 is considered "good," and 0.8-0.9 "very good" (21) . The calibration and discrimination analyses were conducted using the RMAP 0.03-01 (https://gailg.github.io/ rmap/) algorithm in the R environment and Stata version 14 (Stata Corporation, College Station, TX). All other analyses were conducted using SAS version 9.4.
Results
Of the 15 732 eligible women, 619 (3.9%) were diagnosed with invasive BC and 415 (2.6%) died without a BC diagnosis during the 10 years of follow-up. An additional 6078 were alive and without invasive BC with less than 10 years' follow-up (and were censored at date of last follow-up), leaving 8620 women unaffected with invasive BC at 10 years. Table 1 shows the distribution of risk factors for each outcome group. Most women (62.3%) were younger than 50 pittmandarryl@example.org. There were 584 BRCA1-mutation carriers (3.7%) and 491 BRCA2-mutation carriers (3.1%) identified in the study sample. Of the total sample, 21% reported use of HRT.
Results
The calibration results are summarized in 1.24) . A sensitivity analysis excluding 407 women who had taken tamoxifen or been on a chemoprevention trial prior to their BC diagnosis did not appreciably change these results (data not shown). Figure 3A shows that, for the overall cohort, iPrevent was well calibrated overall except for those in the highest decile of risk, where iPrevent overpredicted risk (mean 10-year iPreventexpected risk of 19.1% compared with a mean observed risk of 14.7%). There were similar findings for all subgroups except for women younger than 50 years at baseline, where calibration was good across the spectrum of risk (Figure 3, C , E, G, and I), including the highest-risk decile.
Results
When the analyses were repeated using the same switching algorithm for iPrevent ( Figure 2 ) but with IBIS v8.0b instead of IBIS v7.02, the calibration improved overall (E/O ¼ 1.06, 95% CI ¼ 0.98 to 1.15; Table 2 ) and for noncarriers and both age subgroups, but particularly for women 50 years and older (lower ECIs; Table 2 ). Calibration for BRCA1-and BRCA2-mutation carriers was unchanged because the iPrevent switching algorithm is constrained to choose BOADICEA as the risk estimation model for mutation carriers (Table 2 and Figure 3 B, D, F, H, and J). Regarding discriminatory accuracy, the C-statistic for the entire sample was 0.70 (95% CI ¼ 0.68 to 0.73), 0.74 (95% CI ¼ 0.71 to 0.77) for women younger than 50 years at baseline, 0.63 (95% CI ¼ 0.59 to 0.66) for those 50 years and older at baseline, and 0.59 (95% CI ¼ 0.53 to 0.64) for BRCA1-and BRCA2-mutation carriers (combined) and 0.65 (95% CI ¼ 0.63 to 0.68) for noncarriers ( Figure 4 and Table 2 ). The discriminatory accuracy did not change appreciably when analyses were repeated using the same switching algorithm for iPrevent (Figure 2 ), but incorporating IBIS v8.0b instead of IBIS v7.02 (Table 2) .
Discussion
Accurate estimation of a woman's personal BC risk facilitates the use of evidence-based management strategies appropriate for her risk level and allows calculation of the absolute riskreduction benefit from preventive interventions.
Discussion
In this validation study, based on 10-year BC risk estimates, iPrevent was well calibrated for women younger than 50 pittmandarryl@example.org. This is an important group of women, for whom other relatively user-friendly risk estimation tools, such as the Gail model, do not perform well (8) . The calibration of iPrevent for women 50 years or older and for BRCA1-and BRCA2-mutation carriers was generally good, except for those in the top decile of risk, where it tended to be overestimated. Specifically, iPrevent tended to overestimate risk for women 50 years and older with an assigned 10-year risk no less than 14.3% and for BRCA1-and BRCA2-mutation carriers with an assigned 10-year risk no less than 35.7%. The overestimation of risk found in this study for the highest-risk decile is consistent with our previous study, using the same sample, which found similar findings for the IBIS and BOADICEA models when used separately (7). Uptake of chemoprevention after baseline does not appear to explain the overestimation of risk; in our sensitivity analyses that excluded women who took chemoprevention medications prior to their first BC, the overall inferences were the same. Similarly, we do not believe the overestimation is explained by uptake of bilateral oophorectomy after baseline because we have recently shown that premenopausal bilateral oophorectomy did not reduce BC risk in this cohort (18) . We also censored women at bilateral mastectomy. Regardless, the extent of overestimation is unlikely to be of clinical importance because the actual 10-year BC risks for these women substantially exceed thresholds for intensified screening and medical prevention (and for mutation carriers, risk-reducing mastectomy). *Noncarriers are defined as women not known to be BRCA1-or BRCA2-mutation carriers, so they include tested and untested women. BMI ¼ body mass index. †P value was also <.001 after adjusting for pittmandarryl@example.org.
Discussion
Therefore, the overestimation would be unlikely to lead to an inappropriate change in their clinical management. Importantly, when we used the iPrevent switching algorithm with IBIS v8.0b (rather than v7.02), iPrevent was well calibrated overall for all subgroups. Given that mammographic density and SNP data were not available for this validation study, the improved calibration, seen predominantly for women 50 years and older, suggests that the improved performance might be explained at least in part by the different treatment of HRT model parameters between the two versions of IBIS. HRT use has declined dramatically since the 2002 release of the Women's Health Initiative findings (24) , and the improvement in calibration using IBIS version 8.0b rather than IBIS v7.02 might be less relevant to a contemporary sample of women who may be less likely to use HRT (the vast majority of the baseline data in our study were collected prior to 2002). Nevertheless, using either version of IBIS, the iPrevent algorithm provides good calibration for women younger than 50 years.
Discussion
iPrevent was found to have good discriminatory accuracy (Cstatistic ¼ 0.70, 95% CI ¼ 0.68 to 0.73) overall and with Cstatistics ranging from 0.59 to 0.74 for the subgroups. The overall discriminatory accuracy of iPrevent compares favorably with those found in our prior study of the Gail, BRCAPRO, BOADICEA v3, and IBIS v8.0b risk estimation models, based on the same sample. In that prior study, the C-statistics for the overall sample ranged between 0.60 (95% CI ¼ 0.58 to 0.62) for the Gail model and 0.71 (95% CI ¼ 0.69 to 0.73) for the IBIS v8.0b model, with a C-statistic of 0.70 (95% CI ¼ 0.68 to 0.72) for BOADICEA v3 (7). However, the similar C-statistics for iPrevent, IBIS, and BOADICEA imply that the internal model-switching algorithm programmed into iPrevent does not substantially improve the discriminatory accuracy of the risk estimates provided by either model alone. Nevertheless, iPrevent has the advantage of a user interface that has been designed for use both by consumers and clinicians, and iPrevent provides tailored risk management advice, which the other tools do not provide.
Discussion
iPrevent was developed in response to qualitative research conducted with Australian clinicians and women (25) (26) (27) . They indicated that an evidence-based, computerized tool to facilitate collaborative assessment and management of BC risk by women and their health-care providers would address a major area of need (25) (26) (27) , especially if the tool could be used by women and printed out prior to the consultation (to save time). iPrevent currently uses Australian guidelines to determine the risk management information that is provided to women at each risk level, but future iterations of iPrevent could be made country specific with modifications to align the risk management output with local guidelines (11, 12, (28) (29) (30) . A recent review identified 16 published BC risk estimation models (31) , including IBIS and BOADICEA, but few of those models couple risk management to risk estimation, and studies have demonstrated that clinicians have difficulty applying existing risk models in clinical practice (32) . There are some online risk management tools that enable users to receive information about their risk with suggestions on how to modify it, such as Cancer Australia's Familial Risk Assessment-Breast and Ovarian Cancer tool (33), the Pink Hope Know Your Risk tool (34) , and the Washington University Your Disease Risk tool based on the Nurses' Health Study (35) . However, these models have either not used pedigree models and therefore are not applicable across the spectrum of risk (35) and/or the underlying risk estimation models that the tools use are not published, and there is no information available regarding the accuracy of the risk estimates they provide (33, 34) .
Discussion
Strengths of this validation study include its large sample size with comprehensive and systematically collected baseline risk factor assessment and long follow-up (average more than 10 years), resulting in a large number of incident BCs. A limitation of our study was that most participants were non-Hispanic white. The IBIS and BOADICEA models that underpin the iPrevent risk prediction do not account for differences in underlying BC incidence rates by race or ethnicity, so further validation using racially diverse cohorts is important. Another potential limitation is that almost all women in our study had at least some family history of BC, so further validation using a cohort of women without a family history of the disease would be valuable. In addition, our study sample lacked data on mammographic density and SNP-based polygenic risk, which are included in IBIS v8.0b and have been shown to enhance BC risk prediction (36, 37) , thus our study may underestimate the accuracy of BC risk estimates that would be provided if IBIS v8.0b were incorporated into iPrevent in the future.
Discussion
A key advantage of iPrevent is that it can easily be used to apply relative risk for preventive factors through multiplying the absolute risks from the base models by the relative risk, assuming there is no interaction between the relative risk factor and the absolute risk like we have shown for body mass index (38) . iPrevent currently does this for tamoxifen and riskreducing surgery, but it can easily be adapted to other risk factors that are not included in the existing risk models. iPrevent uses IBIS v7.02, but IBIS v8.0b, which includes mammographic density, was released in September 2017. Adding mammographic density and SNP data has been shown to make small improvements to the performance of the IBIS model (36) . The addition of SNP data also improves BC risk stratification (37) . iPrevent uses BOADICEA v3, which does not consider the effects of mutations in genes other than BRCA1 and BRCA2. Potential iPrevent users are warned not to use iPrevent if they have a mutation in another BC predisposition gene. BOADICEA v4 (39) includes the effects of PALB2, CHEK2, and ATM loss-of-function mutations. Pathogenic mutations in these genes are rare, so our study is likely to have included few such women. BOADICEA v5 has recently been published and includes the associations with a 313-SNP polygenic risk score, other lifestyle and hormonal risk factors, and mammographic density (40) . Future iterations of iPrevent will consider these recent developments and could also potentially incorporate other new risk factors such as sex hormone levels (41) and double-strand DNA break repair phenotype (42) .
Discussion
A major advantage of iPrevent over other risk estimation tools is that it provides personalized information on BC risk management. Another advantage is that the decision about which BC risk estimation model to use is automated, relieving the user of the need for expert knowledge of the performance characteristics of all the relevant risk estimation models. This enables use of iPrevent by women themselves, or their primary care physicians. Future iterations of iPrevent could include IBIS v8.0b rather than v7.02 because, as we have demonstrated, the former improves calibration, even when mammographic density and polygenic risk are not available. As risk estimation models evolve and improve, iPrevent can be updated to integrate the best performing future models. Although initially designed for use in Australia, future versions of iPrevent could be country specific, aligning risk management information with the guidelines of the country in which iPrevent is being used. Clinical decision support tools like iPrevent may help achieve better precision prevention and screening and ultimately achieve the critical goal of reducing BC incidence and mortality. South Wales, Victoria, Tasmania, and South Australia; and

{'paper_id': '215450761', 'title': 'Accuracy of Risk Estimates from the iPrevent Breast Cancer Risk Assessment and Management Tool', 's2_url': 'https://api.semanticscholar.org/CorpusID:215450761', 'source': 'S2ORC'}