Na-Young Jeong,Hyesook Park,Sanghoon Oh,Seung Eun Jung,Dong-Hyun Kim,Hyoung-Shik Shin,Hee Chul Han,Jong-Koo Lee,Jun Hee Woo,Byung-Joo Park,Nam-Kyong Choi
A framework for nationwide COVID-19 vaccine safety research in the Republic of Korea: the COVID-19 Vaccine Safety Research Committee
Committee,COVID-19,Research,Safety,Vaccines
With the introduction of coronavirus disease 2019 (COVID-19) vaccines, the Korea Disease Control and Prevention Agency (KDCA) commissioned the National Academy of Medicine of Korea to gather experts to independently assess post-vaccination adverse events. Accordingly, the COVID-19 Vaccine Safety Research Committee (CoVaSC) was launched in November 2021 to perform safety studies and establish evidence for policy guidance. The CoVaSC established 3 committees for epidemiology, clinical research, and communication. The CoVaSC mainly utilizes pseudonymized data linking KDCA’s COVID-19 vaccination data and the National Health Insurance Service’s claims data. The CoVaSC’s 5-step research process involves defining the target diseases and organizing ad-hoc committees, developing research protocols, performing analyses, assessing causal relationships, and announcing research findings and utilizing them to guide compensation policies. As of 2022, the CoVaSC completed this research process for 15 adverse events. The CoVaSC launched the COVID-19 Vaccine Safety Research Center in September 2022 and has been reorganized into 4 divisions to promote research including international collaborative studies, long-/short-term follow-up studies, and education programs. Through these enhancements, the CoVaSC will continue to swiftly provide scientific evidence for COVID-19 vaccine research and compensation and may serve as a model for preparing for future epidemics of new diseases.
A framework for nationwide COVID-19 vaccine safety research in the Republic of Korea: the COVID-19 Vaccine Safety Research Committee

With the introduction of coronavirus disease 2019 (COVID-19) vaccines, the Korea Disease Control and Prevention Agency (KDCA) commissioned the National Academy of Medicine of Korea to gather experts to independently assess post-vaccination adverse events. Accordingly, the COVID-19 Vaccine Safety Research Committee (CoVaSC) was launched in November 2021 to perform safety studies and establish evidence for policy guidance. The CoVaSC established 3 committees for epidemiology, clinical research, and communication. The CoVaSC mainly utilizes pseudonymized data linking KDCA’s COVID-19 vaccination data and the National Health Insurance Service’s claims data. The CoVaSC’s 5-step research process involves defining the target diseases and organizing ad-hoc committees, developing research protocols, performing analyses, assessing causal relationships, and announcing research findings and utilizing them to guide compensation policies. As of 2022, the CoVaSC completed this research process for 15 adverse events. The CoVaSC launched the COVID-19 Vaccine Safety Research Center in September 2022 and has been reorganized into 4 divisions to promote research including international collaborative studies, long-/short-term follow-up studies, and education programs. Through these enhancements, the CoVaSC will continue to swiftly provide scientific evidence for COVID-19 vaccine research and compensation and may serve as a model for preparing for future epidemics of new diseases.

With the protracted coronavirus disease 2019 (COVID-19) crisis that began in December 2019, vaccines were hailed as a key to emerging from the pandemic and returning to normalcy. Through enormous efforts, vaccines were developed and approved in record time and quickly rolled out to the global population. However, the unprecedented speed of development and emergency use authorization, as well as the fact that some of the newly developed COVID-19 vaccines were based on innovative and unfamiliar platforms, raised safety concerns among experts and the general public. In Korea, COVID-19 vaccines were first introduced in late February 2021. Currently, Korea has one of the highest vaccination rates in the world, with 87.1% of the entire population being fully vaccinated (as of November 18, 2022). Korea carries out vaccine safety monitoring through passive and active surveillance. For passive surveillance, safety data based on vaccine adverse events (AEs) reported by healthcare professionals, vaccinated individuals, and vaccine manufacturers are collected and analyzed, whereas active surveillance utilizes big data such as the vaccination registry, health insurance claims, and electronic medical records (EMRs) to proactively investigate selected population groups for the pre-emptive monitoring of AE cases [1,2]. In the United States (US), the Vaccine Adverse Event Report System serves as a passive vaccine surveillance system that is jointly operated by the Centers for Disease Control and Prevention (CDC) and the Food and Drug Administration (FDA) [3]. For active surveillance, V-Safe, a mobile phone-based system that collects individual AE reports, is operated by the CDC [4]. Once cases are collected, an EMR-based monitoring system called Vaccine Safety Datalink enables each participating site to prepare computerized data files by utilizing a standardized data dictionary for active monitoring [5,6]. The US FDA also established an active surveillance system called the Biologics Effectiveness and Safety System under the umbrella of the Center for Biologics Evaluation and Research. The FDA has used large-scale health insurance claims data and electronic health records to build a protocol for assessing the safety and effectiveness of COVID-19 vaccines [7,8]. The European Medicines Agency has a passive surveillance system called EudraVigilance [9], which collects and analyzes individual cases of suspected AEs after vaccination. For active surveillance, it relies on the Vaccine Monitoring Collaboration for Europe (VAC4EU) [10]. The United Kingdom operates its own existing program, called the Yellow Card Scheme, to collect suspected AEs in relation to drugs, vaccines, and medical devices. For COVID-19 vaccine monitoring, they have added the Coronavirus Yellow Card, a dedicated system for reporting only COVID-19-related vaccine side effects [11,12]. In Korea, since the rollout of the COVID-19 vaccine program, the Korea Disease Control and Prevention Agency (KDCA) has classified AEs into non-serious and serious cases for analysis based on patients’ status at the time of reporting. The results are disclosed on its website every week. In addition, Korea operates the national COVID-19 Vaccine Injury Compensation System for AEs that occur after vaccination. While reviewing requests for compensation, the KDCA saw a growing need to establish safety evidence, focusing on the domestic population. As the demand to generate evidence for vaccine surveillance intensified, the KDCA commissioned the National Academy of Medicine of Korea (NAMOK) to gather experts from medical, pharmaceutical, and healthcare sectors and independently assess the relationship between AEs and COVID-19 vaccines. As a result, the COVID-19 Vaccine Safety Research Committee (CoVaSC) was launched on November 12, 2021 to perform a wide range of COVID-19 vaccine safety studies assessing the relationship between AEs and vaccines in order to establish evidence for policy guidance. Against the backdrop, this paper intends to provide a brief explanation of the CoVaSC, including its organizational structure, activities, major research processes, and findings.

Under the leadership of committee president and assistant administrator, the CoVaSC established 3 committees for epidemiology, clinical research, and communication, as well as 13 sub-committees (Figure 1). The Epidemiology Committee uses COVID-19 vaccine AE reports to monitor and detect safety signals. It also utilizes linked KDCA–National Health Insurance Service (NHIS) data to establish research plans, conduct statistical analyses of the association between COVID-19 vaccines and AEs, and perform observed-to-expected ratio analysis for each target disease. The Clinical Research Committee is in charge of developing operational definitions, creating research protocols together with the Epidemiology Committee, and conducting literature reviews on target diseases and causality assessments based on the results of relationship analyses. The Clinical Research Committee has brought clinical experts on board to form ad-hoc committees for diverse diseases and take the lead in research activities. The Communication Committee develops strategies for communication with healthcare professionals and the public. It also creates press releases and organizes forums to release CoVaSC research findings. A total of 22 members have contributed to CoVaSC research, together with 159 experts in advisory groups.

The CoVaSC has a broad set of roles, including basic surveys, research plan development, data analysis, and reporting (Figure 2). To ensure the transparent sharing of research outcomes with healthcare professionals and the general public, it organizes forums and briefings. The major contents of CoVaSC research include (1) domestic and international trends in causality assessment and the results of literature reviews, (2) monitoring of AE reports and signal detection, (3) statistical analysis of domestic data related to COVID-19 vaccines, (4) causality assessment between COVID-19 vaccination and AEs, and (5) communication with healthcare professionals and the public through regular forums. Promoting close cooperation with the KDCA and NHIS is crucial for the success of CoVaSC research since those 2 organizations are major resources for research data. The KDCA, for example, provides access to its vaccination database, which encompasses healthcare personnel’s reports on AEs in relation to COVID-19 vaccines and COVID-19 vaccination records of the entire population in Korea. The NHIS has an enormous database of health insurance claims, which are generated when healthcare facilities claim insurance reimbursements for medical services that they provide to patients, which covers the entire Korean population of about 51 million. The NHIS links its database with that of the KDCA and provides a data analysis room for researchers to use. The CoVaSC, KDCA, and NHIS hold regular monthly meetings to share their research findings and foster collaboration.

Since the launch of the COVID-19 vaccine program in Korea on February 26, 2021, the KDCA has collected vaccination data from the entire population, and details of adverse reactions have been reported by healthcare clinics and medical institutions in accordance with the Infectious Disease Control and Prevention Act. In addition, in June 2000, Korea started to register immunization records electronically under the National Immunization Program (NIP), and since 2002, all the immunization records at local healthcare centers and private clinics have been kept in the KDCA’s electronic system [13]. The records of COVID-19 vaccines are managed separately from the NIP, and various information, including the vaccination date, vaccine dose, and type of vaccine, is electronically stored. Regarding AE reports, information on the types of AEs, date of onset, and diagnosis and prognosis are collected. These data enable potential safety signals to be detected early and cost-efficiently through AE monitoring.

Since Korea has a health insurance system with universal coverage, all medical services provided to the public are linked with insurance claims data. After providing medical services, healthcare institutions make claims for service fees to the NHIS (the insurer), and insurance claims data are generated in the process. As NHIS manages enrolled and insured individuals and pays costs based on the billing records, its database has accumulated a vast array of data, including insurance eligibility, medical services and prescription records, details of health screening, and information on healthcare institutions [14]. The NHIS claims database provides information on the medical services that have been offered due to AEs after COVID-19 vaccination. The CoVaSC received pseudonymized data that connected KDCA’s COVID-19 vaccination data and NHIS's database. The CoVaSC researchers could access the database to perform analyses at NHIS’s data analysis center. The data were periodically updated during the research period to keep abreast with the latest trends.

Figure 3 shows a general overview of CoVaSC’s research process.

In consultation with the KDCA, the CoVaSC first created a list of adverse events of special interest (AESIs), for which safety issues have been raised in relation to COVID-19 vaccines. In order to set priorities for causality analysis, the CoVaSC conducted a survey in December 2021 to weigh epidemiological and clinical significance and public interest among different research candidates. For hematologic disorders, the incidence rate per 100,000 was first calculated for each disease to identify disorders with an observed occurrence that is significantly higher than expected. Table 1 shows the final priority list of diseases for causality assessment. After defining the target diseases, an ad-hoc committee was established for each target disease, which comprised clinical and epidemiological specialists in the target disease. After finalizing the target diseases, a kick-off meeting of each relevant ad-hoc committee was held to discuss research plans for causality evaluation between the vaccine and the target disease.

Prior to the official start of the research, protocols were written, revised, and reviewed for each target disease. These research protocols described the purpose, duration, and subjects of research, exposure information, the operational definition of the target disease, study design, and statistical analysis methods. Clinicians from each ad-hoc committee devised an operational definition of the related disease by applying diagnosis codes based on the International Classification of Diseases, 10th Revision, procedure/surgery codes, and criteria to exclude underlying diseases. The protocols were first drafted by epidemiologists and then reviewed by the clinicians in the relevant ad-hoc committees for revision and finalization.

Since the CoVaSC relies on a huge amount of data, such as COVID-19 vaccination records, insurance claims data, and AE reports of the entire Korean population, all the missing values and outliers have to be dealt with to extract clean data for the analysis dataset. For example, there were cases where 1 of the 2 vaccination records was missing for fully vaccinated people, or the same vaccination was recorded redundantly, which should have been excluded. Once the data mining process was completed, demographics and vaccination status were reviewed for a technical analysis, and the associations between AEs and COVID-19 vaccines were investigated.

After analyses were completed, study reports were created, including tables and figures to reveal any associations that were found. To perform causality assessments between COVID-19 vaccines and AEs, CoVaSC referred to the Committee of US Surgeon General Criteria [15] and the Bradford-Hill Criteria [16], the 2 globally recognized criteria for causality investigations. The ad-hoc committee of the relevant disease then compiled the analysis results, as well as evidence from Korea and abroad, for a comprehensive review. After a discussion among the ad-hoc committee members, the final decision was made on whether the case fulfilled the causality criteria.

The CoVaSC created press releases and held forums to share its research outcomes with healthcare professionals and the general public. Epidemiologists on the Epidemiology Committee announced the results of epidemiological studies, and clinicians on the Clinical Research Committee shared the results of causality assessments. The CoVaSC held 4 forums by the end of 2022. The first forum briefly explained the purpose and methodology of its research, and the other 3 were held to share research findings. The first forum was held on November 26, 2021 to introduce the CoVaSC, its methodology for epidemiological research, and the criteria that would be used for causality assessment. During the second forum on March 4, 2022, the results of safety monitoring and causality assessments between COVID-19 vaccination and AEs were announced, focusing on death, myocarditis/pericarditis, acute myocardial infarction, and stroke. In the subsequent third forum on May 12, the study findings regarding Guillain-Barré Syndrome, Miller–Fisher Syndrome, heart failure, aortic dissection, acute transverse myelitis (ATM), and acute disseminated encephalomyelitis were released. The fourth and final forum on August 11 showed the results of causality assessments for thrombosis-related diseases, such as deep vein thrombosis, cerebral venous sinus thrombosis and abnormal uterine bleeding (AUB), and presented the causality assessment guidelines for COVID-19 vaccine safety studies. The forums were conducted online and broadcast through YouTube to reach the broader public.

To detect signals of vaccine-related adverse reactions based on the monitoring of AE reports, the research used 2 well-known disproportionality analysis indices for data mining: (1) the proportional reporting ratio (PRR) and (2) reporting odds ratio (ROR). The PRR is calculated by dividing the percentage of a specific AE of a specific COVID-19 vaccine versus the percentage of the same AE from a different COVID-19 vaccine, whereas the ROR is calculated by dividing the odds of AE caused by a specific COVID-19 vaccine by the odds of AE from a different vaccine. Using these 2 indices, the committee explored different combinations of vaccines and AEs to identify adverse reactions that were more frequent than the pre-defined values. Then they observed them as signals to identify statistical associations between a particular AE and a particular COVID-19 vaccine.

For the diseases selected for the association analysis, an observed-to-expected ratio analysis was performed that referred to the pre-COVID occurrence rate to predict the expected occurrence rate after the vaccination against COVID-19. On this basis, the difference between the expected and the observed incidence rates was evaluated. For this study, the monthly incidence rate of each target disease was calculated for 10 years before the COVID-19 vaccination program. The prediction model was produced based on the observed incidence rate using an auto-regressive integrated moving average (ARIMA) model. A seasonal ARIMA model was applied in order to reflect changes in demographics and medical behaviors over time. If the observed value deviated beyond the 95% confidence interval of the predicted value, Poisson regression was conducted.

A cohort study was designed to compare the incidence rate of relevant AEs between the groups with and without vaccine exposure. The incidence rate and the relative and absolute risk rates of relevant AEs were assessed within the follow-up period in the exposed and the non-exposed groups. Although selecting the most appropriate comparator is critically important for cohort studies, it was a herculean task to define the non-exposure group for COVID-19 vaccines due to the overwhelmingly high vaccination rate. Therefore, the research used people who had received influenza shots, for example, as an active control group or people with the same demographic profile and comorbidity index score as a historical control group to analyze outcome variables, particularly death.

If the process of selecting subjects is likely to trigger high selection bias, a self-controlled study design can be applied, which sets an individual patient as a control against himself or herself to address time-invariant confounders. In vaccine safety research, self-controlled case series (SCCS) and self-controlled risk interval (SCRI) designs are mainly used as self-controlled studies. An SCCS is an epidemiological study design that uses patients who experienced the outcome variables during the observation period to specify the risk window during which each individual is potentially at a higher risk of developing AEs. The remaining non-risk period is set as the control window to compare the AE incidence rate between the risk window and the control window [17,18], whereas an SCRI investigates people who have been vaccinated to compare the AE incidence rate between the pre-defined risk period and the control period before or after vaccination [19]. A major difference between the 2 research designs is that an SCCS defines research subjects based on the occurrence of the outcome variables. After specifying the risk window, it defines the rest of the entire research period as the control window. However, an SCRI analyzes people who have been vaccinated and sets a certain period as the control window; therefore, the observation span of an SCRI is shorter than that of an SCCS.

The CoVaSC analyzed the association of AEs likely to occur after COVID-19 vaccination based on the monitoring of AE reports. For AE report monitoring, the committee studied various subgroups defined according to age, sex, and vaccine to describe the status and trend of reporting. It also explored AE signals for different types of COVID-19 vaccines. The CoVaSC conducted diverse analyses on 15 different AEs to estimate their relationship with COVID-19 vaccines (Table 2). Among the diseases where statistical associations were found, myocarditis, pericarditis, and AUB furnished evidence to guide policies. On March 4, 2022, myocarditis was announced to have an association with COVID-19 vaccines. Accordingly, the COVID-19 Vaccine Injury Compensation Committee acknowledged a causal relationship between myocarditis and the mRNA vaccine and applied the decision retroactively. This decision opened the door for those whose previous compensation requests were rejected on the ground of insufficient evidence to receive compensation with no strings attached. For pericarditis not accompanied by myocarditis, based on the finding released on May 12, 2022, the decision was reverted from "inadequate” to “adequate” evidence for its causality with COVID-19 vaccines. This decision also applied retroactively, but with the condition that those who had yet to apply for compensation should submit an application. As of August 16, 2022, AUB was added to the list of diseases with a suspected association (but with insufficient evidence for causality) after releasing relevant research results.

Since Korea introduced the COVID-19 vaccination program without having a monitoring and surveillance system in place, the CoVaSC was established in 2021 to conduct scientific safety assessments of adverse reactions to COVID-19 vaccines based on the available domestic data. As such, the CoVaSC has carried out a diverse range of research, and its research findings have contributed to policy-making and scientific studies. Fifteen diseases were investigated for their associations with COVID-19 vaccines. Victims of AEs for which the CoVaSC suggested potential causality received compensation, and some AEs were added to the list of diseases with a potential association with COVID-19 vaccines. However, due to the limitations of the available data at the time of the initial research, the analyses of some diseases, such as ATM and deep vein thrombosis, were postponed. Various factors have enabled such large-scale, comprehensive research. First, databases played a significant role. The availability of population-wide data, including vaccination records and medical use data, provided a significant boost to the CoVaSC. The KDCA-NHIS linked data offered a rich source of information to conduct association studies. The data have been constantly updated to stay relevant and up-to-date, thanks to well-established governance and close collaboration between the KDCA and the NHIS. Second, the participation of experts from various fields warrants mention. AESIs are very diverse, requiring input from clinical experts in various specialties along with experts in epidemiology, healthcare, and statistics. Therefore, the CoVaSC established Epidemiology, Clinical Research, and Communication Committees and efficiently assigned them different roles. In addition, professional advisory groups and ad-hoc committees for different target diseases were formed in order to ensure efficient and highly coordinated research. By assigning experts to the right place while promoting collaboration, the CoVaSC was able to produce solid research results in a relatively short period of time. Still, there is room for improvement. It is undeniably true that AE reports, vaccination data, and health insurance claims data provide a valuable source of information, yet they have the potential to trigger bias. AE reports are about suspected, not confirmed cases; therefore, they must be linked with clinical data such as EMRs to cross-check whether the diagnosis is correct and appropriate. For insurance claims data, clinical experts devised operational definitions to estimate statistical associations, but often without a full understanding of the actual patients who experienced these conditions. For example, a large number of victims complaining of chest pain were diagnosed with pericarditis, but this diagnosis later turned out to be incorrect, which exaggerated the number of pericarditis occurrences and caused difficulty in accurately estimating the rate. Furthermore, the process of submitting, reviewing, and paying out of health insurance claims caused a time lag in the data, which may compromise the accuracy of real-time analysis. In order to overcome these issues, the CoVaSC plans to develop methods of improving diagnostic accuracy, such as conducting EMR reviews for diseases with extremely low occurrence rates or with low diagnostic accuracy. In addition, for diseases with high clinical significance but low incidence, it plans to establish a network of hospitals to promote joint, collaborative research. The CoVaSC launched COVID-19 Vaccine Safety Research Center in September 2022 and reshuffled the organization from the previous 3-committee system to a 4-division structure, which includes the following divisions: (1) Epidemiological Research, (2) Clinical Research, (3) Communication and Education, and (4) International Cooperation. Under these 4 divisions, there are 7 teams for epidemiological research, 7 teams for clinical research, 1 team each for communication and education, and 1 international cooperation team. Building on existing AE monitoring and causality assessment, the CoVaSC plans to carry out diverse research with the newly established COVID-19 Vaccine Safety Research Center, including international studies, long-/short-term follow-up studies, the construction of a nationwide hospital research network, and educational programs. In particular, with the enactment of the Public Notice on the Designation of the Assigned Agencies to Perform COVID-19 Vaccine Safety Researches & Investigations on November 4, 2022, the commissioner of the KDCA requested the NAMOK to engage in COVID-19 vaccine safety studies and surveillance research, which provides a solid foundation to operate the COVID-19 Vaccine Safety Research Center. To further improvement of the causality assessment, the CoVaSC applied reports and frameworks from the National Academy of Medicine of the US. Based on this experience, it decided to use both epidemiological and mechanistic approaches for future studies to draw conclusions on the associations between AEs and COVID-19 vaccines. To promote international research, the CoVaSC has arranged several international conferences and webinars with the US CDC, the National Center for Immunization Research and Surveillance from Australia, and the World Health Organization to have concrete discussions on study designs and statistical analysis methods. Starting from its collaboration with the Global Vaccine Data Network, the CoVaSC has pursued broader opportunities to work together with other international institutions to perform joint research on the safety assessment of COVID-19 vaccines. In addition, for diseases difficult to analyze with insurance claims data (for instance, if a disease is not reimbursable or patients rarely visit hospitals for the disease), the CoVaSC plans to develop tools to conduct surveys for short- and long-term follow-up. It will also launch an educational training program for clinicians and the general public to deepen their understanding of terminology and causality assessment methodologies in order to lay the groundwork for effective communication to share research results in the future.

The CoVaSC will continue to serve as an organization that swiftly provides scientific evidence for COVID-19 vaccine surveillance. As time passes, its safety database and evidence will constitute essential sources of information to guide policies and help us tackle public healthcare crises in the future. The CoVaSC’s governance and research processes can also serve as a reference for future research projects.