Abstract
Background

a1111111111 a1111111111 a1111111111 a1111111111 a1111111111
Introduction
The concept of the "flipped classroom" was first implemented in 2012 [1] ; since then, its effect has been demonstrated in various fields of education. Unlike in the traditional classroom, students in the flipped classroom review material and obtain knowledge ahead of class, and engage in group activities or projects relying on that knowledge in class [1] [2] [3] [4] [5] [6] . Effects of flipped learning stemming from this element of active participation of students have been shown in various contexts. Most students have shown higher posttest scores compared with pretest scores and higher student satisfaction with the class when taught in a flipped classroom than in a traditional classroom [3, [7] [8] [9] . It has also been reported, however, that national medical licensing examination scores showed no difference between types of instruction [6, 7] . In one study, students scored higher on a skills test for reading an electrocardiogram in the flipped classroom, and spent more time studying alone before class than those who learned in a lecture-based classroom [10] . Belfi et al. compared pre-and posttest scores among students who underwent training in radiology in the flipped classroom, in the lecture-based classroom, and in self-study, respectively; although students in all three groups showed increase in scores after training, the greatest improvement was in the flipped-classroom students, while the other two groups improved at similar, lower levels [11] .
Introduction
The training paradigm of medical schools is shifting to one of performance-based training, because medical students need to develop competencies in medical care for successful performance of their future responsibilities, in addition to the medical knowledge that doctors must have [2, 12] . Developing competencies requires accumulating hands-on experience treating actual patients and cannot be based on simple knowledge transfer, memorization, or written tests. In order to assess the competencies of graduating medical students, the Republic of Korea began implementing the Objective Structured Clinical Examination (OSCE) and Clinical Performance Examination (CPX) its National Medical Licensing Examination in 2010. The basis for incorporating these tests was the argument proposed in "Miller's Pyramid" that the assessment of "knows" (i.e., assessing knowledge level using a written test) is the most primitive stage of assessment, and that assessment needs to move toward assessing "shows" (i.e., assessing whether the student can demonstrate and implement what he or she learned) [13] .
Introduction
The importance of competency in actual medical treatment is especially high in emergency response, such as response to cardiac arrest, because the seriousness of failure to perform properly under psychological and time pressures in emergency is not mitigated whatever the practitioner's level of abstract knowledge. For this reason, training in emergency medicine is also beginning to gradually implement the flipped classroom; however, its practical effect in emergency medicine contexts is not yet clear. Lew reported that both students and teachers in emergency medicine rated the flipped classroom high in terms of satisfaction with the class, in a session teaching an approach to chest pain [14] . Tan et al. reported similar results for an emergency medicine practicum [15] . Rose et al. found that residents in emergency medicine who had learned pulmonary resuscitation of children through simulation and skills training scored higher on posttest than pretest using multiple-choice questions and on satisfaction [16] .A study on teaching advanced cardiopulmonary life support (ACLS) found that students' scores on various written tests improved when the flipped classroom was used [17] . These studies, however, did not assess students' actual competency.
Introduction
It is important for ACLS to be taught using a performance-based approach and evaluated with competency-based methods as opposed to written tests. However, teaching students the required knowledge and skills by actually allowing them to treat patients with cardiac arrest has many practical and ethical complications. Therefore, the present study investigates the effects of the flipped classroom as a means of teaching ACLS to clinical clerkship students in emergency medicine using a simulation approach.
Methods

Study design and participants
This study was randomized and single blinded. We randomly assigned to experimental and control groups 108 fourth-year students in the College of Medicine at Yonsei University, in South Korea, who were scheduled to be in clinical practice in emergency medicine between March and July 2017. The control group learned ACLS in a traditional lecture-based classroom, and the experimental group in a flipped classroom. We assigned students randomly to groups using computer software in February; of 108 students, 53 were assigned to the traditional lecture-based classroom and 55 to the flipped classroom.
Study design and participants
During the student orientation on the first day of clinical practice for each group, one of the researchers briefed students on the study, handed out the study participation consent forms, asked them to read the form, had a Q&A session with the students regarding the study, and asked for their consent to participate. This researcher was in a position uninvolved in student grades and evaluations. The researcher informed students that their participation would have no bearing on their course grades and that their participation status would not be processed until their course grades had been submitted. Students had at least eight hours to make their decision on participation, and submitted the consent form in a sealed envelope at the end of the practicum period. We received the submitted consent forms after students' final course grades in practical training had been submitted to the College of Medicine, that is, upon completion of practical training for both groups. The faculty members who evaluated the educational effects of the two teaching methods using the simulation were unaware of the status of individual students regarding study participation or of the assigned teaching method. Students who insisted on a particular teaching method or who did not attend the ACLS training classes or the simulation assessment session were excluded from the study.
Study design and participants
The primary outcome was simulation rating scores and the secondary outcome was the students' satisfaction survey. 
Study protocol
Practical training in emergency medicine lasted for two weeks, including a one-hour class on ACLS and two hours of simulation. In the traditional classroom, the instructor gave a onehour lecture using PowerPoint slides and held a Q&A session. In the flipped classroom, the instructor emailed students a PowerPoint file with a recorded explanation three days before the class, to allow them to study in advance; then, in class, students watched a video of an example of ACLS and engaged in group discussion. The video contained a "bad case" scenario developed by the Korean Association of Cardiopulmonary Resuscitation for educational purposes. After watching the video, students were asked to find and correct the errors in the treatment procedure, in teams (3 or 4 students). After the ACLS class, students received two hours' simulation training to reduce potential difference in understanding of the simulation and underwent assessment of the educational effects in a familiar environment. In addition, to maintain objectivity, different faculty members separately conducted classroom instruction, simulation training, and student assessment, and faculty members ensured that all student groups received the same instructional content throughout the study period. On the last day of practical training, students in both groups performed an identical ACLS scenario on a simulation mannequin (Laerdal Medical, Stavanger, Norway), in teams of three or four; the researchers scored their performance as a team using a rating form developed in advance. Students were trained in the simulation in separate rooms with different access routes, to prevent exchange of information on the simulation. In addition, to determine whether any differences in assessment results of the two groups were due to features of that particular simulation, the OSCE and CPX scores that were scored during the emergency medicine clerkship were compared and they also went advanced trauma life support (ATLS) simulation training and assessment, conducted in the same manner as for ACLS.
Study protocol
The simulation rating form consisted of items on individual skills and a global rating. A total of 13 items on skills concerned whether each skill was performed properly; each item scored one point if the skill was demonstrated. The global rating item scored up to five points based on the assessment of overall performance on the simulation. (Table 1 ) Table 1 . Simulation rating form.
Definition

Q1
Did the team use a defibrillator monitor? The highest possible score depended on the scenario: 18 points if fibrillation was necessary and 17 if not. We converted scores to a 100-point scale. Due to the nature of simulation training, scores applied to the whole team instead of each individual. After the assessment, the participants completed a survey to determine their satisfaction with the respective methods of instruction they had experienced by grading between 1 to 10 points by themselves (Table 2) .
Statistical analysis
We calculated sample sizes for a two-sample means test using previous study data of 2 teams (181 students) with a traditional classroom group means of 8.03, SD±1.01, and a flipped classroom group means of 8.72, SD±1.01, in the ECG interpretation score [18] . Based on this data, the sample size was obtained using PASS (version 12, NCSS, LLC, Kaysville, Utah, USA). We estimated that a sample size of 53 pairs (106 participants) would be sufficient to evaluate the primary outcome at a significance level of 0.05 (two-sided) with 80% power, considering a 10% dropout rate.
Statistical analysis
We used R version 3.2.2 (The R Foundation for Statistical Computing, Vienna, Austria) for the statistical analyses. We performed score comparison using t-test and Wilcoxon rank-sum test, and frequency comparison using chi-squared test and Fisher's exact test. For all tests, we defined p-value <0.05 as statistically significant.
Results
Out of 122 practicum students during the study period, 108 participated in the study. The 14 students who did not participate included 13 students who had already received training before preparation for the study was complete, and thus could not consent to participation, and one student who could not participate in clinical practice due to illness. (Fig 1) We present demographic information on study participants in Table 3 . ACLS simulation scores of the students in the flipped classroom were 70.9±10.9, which was higher than those of the students in the traditional classroom (67.1±11.3); however, the difference was not statistically significant (p = 0.339). The difference in student satisfaction as measured on the survey was also statistically insignificant (p = 0.655).
Results
The students in the flipped classroom showed higher scores on the following individual items: checking if the patient is conscious, requesting help or speaker announcement, and properly administering epinephrine. The students in the lecture-based classroom showed higher scores only on checking at 2-minute intervals to see if self-circulation was restored (Table 4) 
Survey Items (grade1-10 points)

R1
It was generally a good class.
R2
I gained a lot of knowledge from this class.
R3
I actively participated in this class.
R4
The amount of time assigned to this class was adequate.
R5
The level of difficulty of this class was appropriate.
R6
I gained sufficient understanding of the content through this class.
R7
This class was interesting to me.
R8
I would recommend this class to other students.
R Sum Sum of R1 through R8
https://doi.org/10.1371/journal.pone.0203114.t002 
Discussion
A variety of efforts are being made to shift the curriculum of medical training from a teachercentered to a learner-centered model. This is because modern medical training is inefficient and inflexible, and focuses on getting good test scores instead of developing professional competencies [2, 19] . The flipped classroom is an effective teaching method for changing students from passive to active learners and creating integrative educational strategies, as the teacher provides students with class materials ahead of time for students to study alone, while classroom activities such as discussion that require students' active participation are the focus during the class [20] . The flipped classroom demonstrates better outcomes than the traditional lecture-based approach for students' comprehension and application of course content [21] . A previous study compared the written test results of practicum students in emergency medicine who had learned about ACLS in a flipped classroom versus those who had learned in a regular classroom using lecture and simulation, and showed that the students in the flipped classroom scored higher than the other students [17] . The present study randomly assigned senior medical students on their practicum in emergency medicine to learn ACLS in two groups, one in a flipped classroom and one in a traditional lecture-based classroom. Competency assessment based on simulation performance results showed no statistically significant difference between groups. However, the observed difference was, although not statistically significant, tending toward 100 points which is important because our checklists were made through the essential process of ACLS. The success of ACLS does not merely depend on any one particular action or process. It depends on the results of the appropriation of the whole action or process.
Discussion
There are other studies that do not demonstrate any effect of the flipped classroom. Chen et al. conducted a systematic review of nine studies that evaluated the flipped classroom in medical training, and reported that its effect on knowledge and skills change was inconclusive, as the statistical data required to estimate effect size could be obtained from only four studies [22] . A study with students in an emergency medicine practicum similarly found that those who had learned in a flipped classroom did not show more positive outcomes than those in a traditional classroom on a multiple-choice test; however, due to the lack of data on how well students had prepared for the class in the flipped classroom, this study's findings may have been limited in value [23] . In another study, on the comparative effects of the flipped classroom for students on their ophthalmology practicum, final test scores showed no significant difference between treatment and control groups [24] . It has even been noted that, although the flipped classroom is an attractive teaching method for residents who do not have enough time for class, actual results show that only satisfaction with the method, not its substantive educational effect, is high [25] . Finally, a study of a neuroanatomy course showed that the flipped classroom evidenced no differences in test scores or in satisfaction from the traditional teaching method, which the investigator of the study interpreted as being because there was too much content to teach [26] .
Discussion
Overall, research that has demonstrated the efficacy of the flipped classroom has been mostly based on the comparison of written test results. Written tests are certainly an easy way to assess students; however, since medical training needs to develop and assess actual practical competencies for medical treatment, it remains important to conduct more studies assessing competencies, whether using simulations or in the actual job setting, instead of simply comparing written test results to evaluate the educational effects of the flipped-classroom method.
Discussion
Our study has some limitations. First, where most previous studies in this area have focused on students' satisfaction and changes in knowledge and skills, and have failed to compare student outcomes in higher domains, for example as presented in Kirkpatrick's classification (e.g., change in behavior, professional competencies, patient outcomes), the present study focused on competencies, teamwork, and problem-solving skills using simulation as an assessment method; however, this study could not compare patient outcomes or long-term behavior change because of the limitations of the simulation and time limitations. Second, we could not find an objective way to measure students' compliance. In a different context, Heitz et al. reported that only 31% of students failed to properly prepare for class [23] . In the present study, it is likely that the discussion format used in the flipped classroom placed pressure on students to prepare for the class; however, there are no relevant data. Third, we could not eliminate the possibility of the cross-contamination issue because students were not bound to their group assignment, and therefore could study with students from the other group. However, within the group, students were trained on simulation in separate rooms with different access routes, to ensure they did not share information on the simulation. Also, we did not reveal the checklists to students to ensure they did not find out what was going to happen before the simulation. Fourth, we have the limitation of being unable to be free of multiple testing issues while verifying each hypothesis for multiple questions.
Discussion
Finally, due to the structure of the medical school practicum, we implemented training for only two weeks, even though there were many required items to teach; consequently, only one hour of the flipped classroom was applied to ACLS training. Thus, the minimum number of hours needed for the flipped classroom to be effective has not yet been established, and further study will be needed on these issues.
Discussion
Nevertheless, it is worth appreciating the fact that almost all students and teachers preferred the flipped classroom, although the studies found only limited or no effects, because motivation is the first step for adult learners to gain educational benefits.
Conclusions
An ACLS competency assessment conducted using a simulation after instruction in ACLS undergone by senior medical students randomly assigned to either a flipped classroom or a traditional lecture-based classroom showed no statistically significant difference between the two teaching methods. 
Supporting information


{'paper_id': '207367707', 'title': 'Flipped-classroom training in advanced cardiopulmonary life support', 's2_url': 'https://api.semanticscholar.org/CorpusID:207367707', 'source': 'S2ORC'}