{'fraction_non_alphanumeric': 0.03914527902048505, 'fraction_numerical': 0.022133270543913352, 'mean_word_length': 4.931913407821229, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 5, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Prognostic Nomogram Based on the Metastatic Lymph Node Ratio for T1-4N0-1M0 Pancreatic Neuroendocrine Tumors After Surgery

Purpose This study aimed to investigate the prognostic significance of the metastatic lymph node ratio (LNR) in patients with pancreatic neuroendocrine tumors (pNETs) and to develop and validate nomograms to predict 5-, 7-, and 10-year overall survival (OS) and cancer-specific survival (CSS) rates for pNETs after surgical resection. Methods The demographics and clinicopathological information of T1-4N0-1M0 pNET patients between 2004 and 2018 were extracted from the Surveillance, Epidemiology and End Results database. X-tile software was used to determine the best cutoff value for the LNR. Patients were randomly divided into the training and the validation groups. A Cox regression model was used in the training group to obtain independent prognostic factors to develop nomograms for predicting OS and CSS. The concordance index (C-index), calibration curves, area under the receiver operating characteristic curve (AUC) and decision curve analysis (DCA) were used to assess the nomograms. Patients were divided into four groups according to the model scores, and their survival curves were generated by the Kaplan–Meier method. Results A total of 806 patients were included in this study. The best cutoff value for the LNR was 0.16. The LNR was negatively correlated with both OS and CSS. Age, sex, marital status, primary site, grade, the LNR and radiotherapy were used to construct OS and CSS nomograms. In the training group, the C-index was 0.771 for OS and 0.778 for CSS. In the validation group, the C-index was 0.737 for OS and 0.727 for CSS. The calibration curves and AUC also indicated their good predictability. DCA demonstrated that the nomograms displayed better performance than the American Joint Committee on Cancer (AJCC) TNM staging system (8th edition). Risk stratification indicated that patients with higher risk had a worse prognosis. Conclusions The LNR is an independent negative prognostic factor for pNETs. The nomograms we built can accurately predict long-term survival for pNETs after surgery.


INTRODUCTION
Pancreatic neuroendocrine tumors (pNETs) are relatively rare tumors that originate from the pancreatic neuroendocrine system (1). According to the US epidemiology survey, its incidence is less than 1 in 100,000 people, representing nearly 10% of all pancreatic tumors (2,3). Owing to the advancement of imaging and endoscopic techniques, the detection rate of pNETs has gradually increased in recent years (4). PNETs are classified as functional and nonfunctional types based on whether they secrete hormones associated with a clinical syndrome, while nonfunctional pNETs account for most of them (4,5). One of the characteristics of pNETs is heterogeneity. Generally, they exhibit indolent clinical features; however, they may become invasive and transform rapidly in some circumstances (6).
Surgical resection is an effective treatment for pNETs without metastases (7). One SEER database study demonstrated that surgery might greatly improve survival compared with nonsurgery interventions (114 months vs. 35 months) (8).
Other studies reported that the 5-year survival rate after surgical resection of pNETs was approximately 80% (6,9). However, due to the rarity of this disease, studies on prediction models for pNETs after surgery are deficient.
Recently, the metastatic lymph node ratio (LNR), defined as the proportion of positive nodes to total examined nodes, has been an important prognostic factor for many tumors. You et al. (10) pointed out that the LNR might predict the prognosis of patients with pancreatic cancer. Zhang et al. (11) demonstrated that the LNR was a strong negative prognostic factor for patients with colorectal cancer. Two other studies also indicated that it was negatively related to the survival of patients with gastric and small intestinal neuroendocrine tumors (12,13). However, there are no reports of nomograms of long-term survival after resection of pNETs that incorporate LNR data.
Therefore, based on the Surveillance, Epidemiology, and End Results (SEER) database, the present study attempted to explore the correlation between the LNR and the prognosis of pNET patients and to construct nomograms to predict 5-, 7-, and 10year overall survival (OS) and cancer-specific survival (CSS) rates for pNETs after surgery.

Data Collection
The SEER database is an authoritative cancer statistics database in the United States, covering approximately 30% of the US population (9). Data were extracted from the SEER database with SEER*Stat Software (version 8.3.9.2), and the Incidence SEER Research Plus Data, 9 Registries, Nov 2020Sub (1975 dataset was selected for analysis (username for login: 15881-Nov2020). Patients diagnosed with pNETs from 2004 to 2018 were identified retrospectively. The corresponding selection formula in the software was as follows: Site and Morphology, "Site recode ICD-O-3/WHO 2008" (Pancreas) and "ICD-O-3 Hist/behav" (8013/3, 8150/3-8156/3, 8240/2, 8240/3-8246/3, 8246/2, 8249/3). The following variables were extracted from the database: patient ID, age, sex, race, marital status, year of diagnosis, primary site, histologic type, grade, diagnostic confirmation, tumor size, T stage, N stage, M stage, American Joint Committee on Cancer (AJCC) TNM staging system, surgery at the primary site, the scope of regional lymph node surgery, radiation recode, chemotherapy recode, regional nodes examined, regional nodes positive, survival months, vital status recode, SEER cause-specific death classification, first malignant primary indicator and sequence number.

Data Processing
Patients conforming to any of the following criteria were excluded: 1) surgery was not performed; 2) no regional nodes examined or the number of regional lymph nodes removed was unknown; 3) the AJCC TNM staging system belonged to M1; 4) not the first malignant tumor or multiple primary tumors; and 5) missing or unknown clinical information.
Age was regarded as a continuous variable, and other factors were treated as categorical variables. Patients who were widowed, divorced, single or separated were considered unmarried. (C25.4, islets of Langerhans), (C25.7, other specified parts of the pancreas), (C25.8, overlapping lesion of the pancreas) and (C25.9, pancreas, NOS) in the primary site were considered other. The staging system was adjusted according to the 8th edition AJCC. OS was defined as survival time until death by any cause or last follow-up, and CSS was defined as survival time until death due to pNETs (9).

Construction and Validation of the Nomograms
The overall patients were randomly divided into a training cohort and a validation cohort at a ratio of 7:3. For the training group, univariate and multivariate Cox regression analyses were used to screen out significant variables. After that, all the independent prognostic factors in multivariate analysis were used to construct nomograms using the package "rms" in R software (14). The validation cohort was used to perform an external validation of the nomograms.
To assess the accuracy of the nomograms, we calculated Harrell's concordance index (C-index) and drew calibration curves of the training and validation groups for internal and external validation (15). The area under the receiver operating characteristic curve (AUC) was also used to evaluate the predictive value (15). In addition, we used decision curve analysis (DCA) to determine the 5-, 7-, and 10-year survival rates of the two groups (16). Furthermore, DCA was applied to compare the AJCC staging system (8th edition), the LNR and the nomograms.

Statistical Analysis
All statistical tests were performed using R statistical software (version 4.1.2, https://www.r-project.org, Vienna, Austria). Categorical variables were expressed as frequencies and percentages. Continuous variables were presented as the mean with standard deviation. The optimal cutoff value for the LNR was determined by the X-tile program (17). Survival curves were generated by the Kaplan-Meier method and analyzed by the logrank test. Variables with P < 0.2 in univariate analysis were subjected to multivariate analysis. P < 0.1 in multivariate analysis was considered clinically significant. Pearson's correlation was performed to detect collinearity among the variables. A correlation coefficient < 0.7 between two variables indicated no multicollinearity (18). Tolerance and variance inflation factor (VIF) were also used to evaluate multicollinearity between variables, with tolerance < 0.1 and VIF > 10 indicating multicollinearity (19). P < 0.05 was considered statistically significant.

Patient Characteristics
A total of 806 patients with T 1-4 N 0-1 M 0 pNETs were enrolled in this study, of which 564 cases were randomly assigned to the training group, while 242 cases were assigned to the validation group ( Figure 1). The clinicopathological characteristics of patients in the 2 cohorts are summarized in Table 1. In the training group, 75 patients (13.3%) died from pNETs, and 23 (4.1%) patients died from other causes, while in the validation group, the numbers were 27 (11.2%) and 5 (2.1%), respectively. The optimal cutoff value for the LNR was 0.16. Therefore, patients were divided into three groups (LNR1: 0, LNR2: ≤ 0.16 and LNR3: > 0. 16). Patients in the high LNR group had poorer OS and CSS than those in the relatively low LNR group (Figures 2A, B).

Variable Screening and Nomogram Construction
For the training cohort, the univariate analysis for OS revealed that age, sex, marital status, primary site, grade, tumor size, T stage, LNR, radiotherapy and chemotherapy were significantly associated with survival. Additionally, age, sex, marital status, primary site, grade, tumor size, T stage, LNR, radiotherapy, chemotherapy and lymph node dissection were significantly associated with CSS. The multivariate Cox regression analyses revealed that age, sex, marital status, primary site, grade, LNR and radiotherapy were significant for both OS and CSS ( Tables 2, 3). Therefore, these factors were included in the construction of the nomograms. Every variable was given a score in these two nomograms. Users could obtain the total score based on the individual scores of those factors and estimate the probability of survival for 5, 7 and 10 years ( Figures 3A, B).  There was no significant correlation between the screened variables for the overall dataset, the training or the validation groups ( Figure 4). Furthermore, the tolerance was >1, and the VIF was <10 for the three groups, proving no collinearity among the factors.

Nomogram Validation
The two nomograms were validated both internally and externally. In the internal validation, the C-index was 0.771 for OS and 0.778 for CSS. In the external validation, the C-index was 0.737 for OS and 0.727 for CSS. The calibration plots were close to the standard curves for OS ( Figure 5A) and CSS ( Figure 6A) in the training group and for OS ( Figure 5E) and CSS ( Figure 6E

Risk Stratification
To determine the performance of the established nomograms in stratifying the risk of pNETs patients, we divided them into four     (Figures 8A, B) and CSS ( Figures 8C, D) were found among the four groups in both the training and validation groups.

DISCUSSION
PNETs are heterogeneous tumors with different prognoses. Therefore, predicting the outcomes of these patients is complicated. In this study, we screened data from the SEER database and studied the factors affecting the long-term survival of T 1-4 N 0-1 M 0 pNET patients after surgery. After that, we constructed nomograms to predict the 5-, 7-, and 10-year OS rates and CSS rates for these patients. Considering the importance of the LNR, we included it in the nomograms. The relatively high C-indices and values of AUC in the training and validation cohorts proved the good clinical predictive ability of the nomograms. The calibration plots also demonstrated their reasonable prediction probability. The LNR has been increasingly recognized as a strong predictor of survival for many neoplasms. Teng et al. (20) noted that the LNR was a significant negative predictive variable for the OS of breast cancer patients. Tol et al. (21) reported that the LNR was an important predictor of poor survival in ampullary cancer. Using SEER data regarding pNETs, Liu et al. (22) suggested that the LNR, but not the total number of examined lymph nodes or lymph node metastasis, was an adverse prognostic factor for OS.   Heidsma et al. (23) pointed out lymph node status (yes/no presence of positive node) was an independent factor to predict 5-year recurrence after resection of grade 1 and 2 nonfunctional pNETs. Ricci et al. (24) indicated that the LNR > 0.07 was associated with lower recurrence-free survival (RFS), whereas Boninsegna et al. (25) identified an LNR > 0.2 as an independent indicator of poor RFS. Another study showed that an LNR ≥ 0.5 was associated with worse CSS and that the LNRbased staging system was better than the 8th edition AJCC N staging (26). Similar to previous reports, this study showed that the LNR was a poor independent prognostic factor for both OS and CSS. Moreover, the LNR, being a ratio, eliminates the variability in lymph node sampling during surgery and is more suitable for evaluation of prognosis. In addition to LNR, several other variables were included in the established nomograms, such as age, sex, marital status, primary site, grade and radiotherapy. Several studies have indicated that age significantly impacts the survival of pNET patients (9,27,28). A similar negative effect was observed in this study. There are two possible reasons for this. First, tumor resistance in elderly patients is poorer than that in young patients due to physical aging; thus, the CSS of these patients is poor (14). Second, there are far more age-related comorbid conditions in elderly patients than in young patients; therefore, the OS of elderly patients is poor. Female patients had more favorable prognoses than male patients, which was in agreement with the conclusion of Miao et al. (29). Marital status was also shown to affect both OS and CSS, consistent with previous reports (29,30). This is because married patients usually have better psychological mentation and socioeconomic status, which may indicate a better prognosis. Similar to a previous study, the primary site of the tumor was associated with survival (29). Furthermore, histologic grade, which is an inherent characteristic of tumors, was a critical factor in predicting the prognosis of pNET patients in this study. Similar results were found in previous reports (9,31,32). Interestingly, this study  showed that radiotherapy had some impact on survival. Iwata et al. (33) pointed out radiotherapy was an effective treatment for local disease control. However, the number of patients in that research was too small. The effect of radiotherapy on pNETs requires large sample studies. For the first time, we developed and validated nomograms to predict 5-, 7-, and 10-year OS and CSS for T 1-4 N 0-1 M 0 pNET patients after surgery. The nomograms included not only clinicopathological factors but also demographic factors and were more precise than the conventional staging system. Furthermore, we compared the nomograms with the AJCC TNM staging system (8th edition). DCA curves proved that our nomograms prediction ability was better than the AJCC TNM staging system. In addition, the nomograms were composed of variables that are readily available in clinical practice. With this easily used system, doctors could evaluate the risk factors for patients more precisely. Risk stratification demonstrated good applicability for patients in different stages. Therefore, patients with high risks of poor prognosis may benefit from adjuvant therapy. These may promote more specialized individualized treatment for this heterogeneous neoplasm.
There were some limitations in this study. First, it was a retrospective study based on the SEER database, and this may lead to the risk of potential selection bias. Second, several important clinicopathological factors, such as carbohydrate antigen 19-9 (CA19-9), Ki-67 index, surgical margin status, chemotherapy regimens and radiation technology, were not included due to the limitation of the SEER database, which might affect the results.
In conclusion, the present study identified that the LNR is an independent prognostic factor for pNETs. We established nomograms based on the SEER database to predict long-term survival for pNETs after surgery. These nomograms could help clinicians make tailored treatment plans for patients.

DATA AVAILABILITY STATEMENT
The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found in the article/Supplementary Material.