{'fraction_non_alphanumeric': 0.035218138735306416, 'fraction_numerical': 0.02216233796403847, 'mean_word_length': 5.222607690083839, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Exploring mobile mixed reality in healthcare higher education: A systematic review

Background: The evolution of technology and simulation has had a significant impact on clinical education. However, it remains grounded in traditional teaching paradigms, limiting potential for enhanced learning. Furthermore, the impact of mixed reality enabled mobile devices remains underexplored. Purpose: The aim of this article was to investigate mobile learning and mixed reality in healthcare higher education. Method: A search of six databases from the earliest available date to 30 February 2018 and a hand search of journals and included studies was performed. Inclusion criteria focused on ‘healthcare’, ‘higher education’, ‘mobile learning’ and ‘mixed reality’. All study designs were included, though they were limited to the English language. The checklist of the Preferred Reporting Items for Systematic Reviews and Meta-Analysis was used as a framework for the review, with included studies critiqued using the mixed methods appraisal tool. Results: The search generated 1484 studies, with 18 meeting inclusion criteria. The majority of studies utilised mobile mixed reality (mMR) for teaching procedural skills with established mobile platforms; anatomy; and clinical assessment. mMR demonstrated benefits in skill competency and knowledge scores when compared to control. Users were favourable towards future use of mMR. Conclusion: While mMR successfully delivered some clinical skills; the pedagogical impact of engagement with higher order clinical reasoning remains a challenge for future studies. Published: 27 November 2018 This paper is part of the special collection  Mobile Mixed Reality Enhanced Learning , edited by Thom Cochrane, Fiona Smart, Helen Farley and Vickel Narayan. More papers from this collection can be found here . Citation: Research in Learning Technology 2018,  26 : 2131 -  http://dx.doi.org/10.25304/rlt.v26.2131


Introduction
Pressures on healthcare higher education (HHE) has led to exploration of different ways to engage students. Pressures include a rise in student cohort numbers to meet industry demand with corresponding reduction in staff to student ratio, classroom space and resources (American Association of College of Nursing 2017). Concurrently, students find it increasingly difficult to balance study, work and life, especially with curricula that are delivered at a certain time, place and mode (Sauve et al. 2018).
The evolution of information technologies has been advocated to assist the pressures on HHE (Houghton et al. 2014). However, despite the advent of clinical simulation, curriculum remains predominantly embedded in behaviourist and teacher-directed learning rather than a student-directed and constructivist pedagogy to develop clinical reasoning and a more active role in learning (Houghton et al. 2014). As access to HHE evolves with the development of low-cost technology and use of high-fidelity clinical simulation, educators are encouraged to consider pedagogy that embraces student-directed and technology-enabled learning (TEL). While no single mode of TEL will meet the needs of all students, a multitude of methods should be considered (Mayer 2014).

Mixed reality, mobile learning and mobile mixed reality
According to Milgram and Kishino (1994), mixed reality (MR) is the convergence of the real and virtual worlds. Mixed reality exists on a reality-virtuality continuum, with augmented reality (AR) and augmented virtuality (AV) in between ( Figure 1). Fidelity in HHE has been defined as 'the degree to which a simulation looks, feels, and acts like a human patient' (Hamstra et al. 2014). The perceived need for high fidelity (closer to real environment) has been regularly debated, though it is limited by expense, on-site equipment and training (Zendejas et al. 2013). However, the value of low fidelity can be advocated for (Hamstra et al. 2014) as long it maintains a relation between functional aspects of the simulation (e.g. tactile, visual and auditory features) and the human functional factors for education (e.g. context and complexity of task, stage of student learning and appropriate pedagogy). Mixed reality offers the opportunity to rethink and design innovation and clinical simulation spaces for learning and teaching in HHE (Magana 2014).
Mobile devices -in particular, smartphones -are increasingly becoming embedded into the socio-cultural activities of modern life (International Telecommunication Union 2015) as sensors and communication technology enable location of affordances nearby and the student's social network (Olsson et al. 2013). Mobile learning (mLearning) has become de rigueur in secondary education (OECD 2015) and is progressively being integrated into higher education to enable flexibility and student-directed learning while providing social interactions using personal mobile devices (Bujak et al. 2013;Crompton 2013  been documented to improve competency of physical tasks, support memory-related activities and develop situated cognition within social, cultural and physical contexts in education (Sharples 2002).
The aim of this review is to explore the integration of mMR informed by learning theory and new pedagogical strategies in HHE. Specifically, it investigates the possibilities presented by the rise of mLearning and MR to enable new pedagogical strategies in HHE.

Methods
This systematic review followed the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) (Moher et al. 2009). A comprehensive literature search was performed on the use of mMR in HHE in the following health and education databases: Scopus; PsychINFO (OVID); ERIC (OVID); Medline (OVID); Web of Science and AMED (OVID). Search terms used to identify articles included the following: 'higher education', 'healthcare', 'mobile learning' and 'mixed reality' (Appendix 1). While 'extended reality' was not included in the search terms, subsets along the reality-virtuality continuum as identified by Milgram and Kishino (1994) were included in the search. Publisher sites were searched for relevant educational technology journals (Appendix 2), and reference lists of included studies were hand-searched. Databases were searched from the earliest available date to 30 February 2018, with only English-language articles included.

Study selection
Studies were imported into EndNote software (Clarivate Analytics) and duplicate articles removed. Two assessors performed a first-level selection based on titles and abstracts by following the inclusion and exclusion criteria independently.
Population-intervention-comparison-outcome (PICO) inclusion criteria were as follows: population -HHE (i.e. undergraduate healthcare education), including, though not limited to, medicine, nursing, physiotherapy, paramedicine, podiatry, midwifery, oral health and occupational therapy; intervention -MR-capable mobile devices (such as mobile phones, tablets or non-tethered head display units; HDUs) for teaching and learning; and outcome -the impact on clinical skill competency, which was defined as measurement of clinical skill ability as determined by a competency checklist. The secondary outcome was reporting of mMR usability. Exclusion criteria were the following: primary or secondary education (Year 1-13) settings, postgraduate or clinical settings; use of laptops, tethered HDUs or simulation suites; did not include AR, AV or virtual environment as a mode of teaching; or were not original empirical study designs.
The full texts of all the included or potentially relevant articles were imported into EndNote. A second-level selection of these articles were performed by the same two assessors independently by reading the full text of the articles and following the same inclusion and exclusion criteria. Inter-rater reliability percentage agreement at second-level selection was 91.30%. Discrepancies were resolved by consensus or when needed by a third reviewer, both at the first and second levels of selection. A PRISMA flowchart describing the number of studies included and excluded at the first and second levels, as well as the reason for their exclusion at the second level of selection, were prepared ( Figure 2). The quality of included studies was determined by the mixed methods appraisal tool (MMAT) (Hong et al. 2018). The MMAT is designed for systematic reviews that include qualitative, quantitative and mixed-methods studies and is efficient as it allows the use of one tool for concomitantly appraising the most common types of empirical studies. After responding to two screening questions, each included study was rated in the appropriate category of criteria as either 'yes', 'no' or 'can't tell'. While calculation of an overall score is discouraged, the MMAT provides a more detailed presentation of the ratings of each criterion to better inform the quality of included studies. The MMAT is based on constructionist theory and has already been used by more than 100 systematic mixed study reviews (http://mixedmethodsappraisaltoolpublic.pbworks.com/). The MMAT inter-rater reliability between the two raters was 99.96% (Appendix 4).

Data collection process
Qualitative data were extracted from the included studies by one reviewer and crosschecked by a second reviewer. As quantitative data was not being collected or analysed, this review did not include items 10-16 on the PRISMA checklist.

Results
The search returned 1480 papers, with four additional papers included after handsearching appropriate journal publication sites and source references (total 1484 articles).

(page number not for citation purpose)
After duplicates were removed, 961 papers were available. After titles and abstracts were screened, 938 papers were excluded. Following the review of 23 papers retrieved for full text evaluation, an additional five studies failed to meet the inclusion criteria; therefore, 18 papers were included in the review. For each included study, study design, healthcare discipline (participants), intervention, measure and key findings were extracted (Table  1). See Appendix 3 for a summary of papers excluded after review of the full text.

Quality
A variety of study designs were employed in the included studies -qualitative studies (n = 1); randomised controlled trials (RCTs) (n = 5); non-RCTs (n = 1); quantitative descriptive studies (n = 3); mixed methods (n = 7); and design-based research (DBR) (n = 1). All studies that included qualitative design performed well on the MMAT; however, those that included RCT in their mixed methods were more variable, with only three meeting all criteria in the MMAT (Aebersold et al. 2018;Cowling and Birt 2018;De Oliveira et al. 2013). The purpose of the four descriptive studies was to establish validity of the TouchSurgery® modules (Khelemsky, Hill, and Buchbinder 2017;Kowalewski et al. 2017;Morone et al. 2017;Sugand, Mawkin, and Gupte 2015), therefore they included survey and cross-sectional analytic study in their design.

Mobile device included
Some mobile devices required Internet connectivity to download the mMR application while others were preinstalled prior to the study commencing. The majority of studies included iPhone and/or Android mobile phones (Albrecht et  Google Cardboard or ColorCross were also used with mobile phones in three studies (Birt et al. 2017a(Birt et al. , 2017bCowling and Birt 2018).

Software
Three studies (Albrecht et al. 2013;Noll et al. 2014Noll et al. , 2017) evaluated a dermatology module on the mARble® platform -a mobile AR-based simulation application used to access interactive text, graphic, audio and video using off-the-shelf AR markers (http://marble-app.weebly.com/). Küçük, Kapakin and Göktaş (2016) used the Aurasma platform to embed AR markers to create a MagicBook® that students accessed on their mobile devices to view the interactive, augmented or superimposed object. One study (Garrett, Jackson, and Wilson 2015) included Layar (www.layar. com) to provide interactive markers for the available educational resources. Six studies (Amer et al. 2017;Bartlett et al. 2017;Khelemsky, Hill, and Buchbinder 2017;Kowalewski et al. 2017;Morone et al. 2017;Sugand, Mawkin, and Gupte 2015) evaluated various modules on the TouchSurgery platform -an interactive AR application that is freely available for download on mobile phones and tablets. The app aims to familiarise the student with surgical procedural steps with a 'tutorial', then a 'test' module to assess understanding of the techniques. TouchSurgery calculates total scores as a percentage of correct decisions, appropriate swipe interactions and the time taken to complete each step.
The remaining studies involved external developers to provide an appropriate mMR platform (Aebersold et al. 2018;Birt et al. 2017aBirt et al. , 2017bCowling and Birt 2018;De Oliveira et al. 2013;John et al. 2015), some of which used 3D printed equipment in the augmented environment (Birt et al. 2017a(Birt et al. , 2017bCowling and Birt 2018).

Benefits of mMR
Reported benefits from the use of mMR in healthcare education included improvement in procedural skills, such as scope and tube placement, and anatomy knowledge (Aebersold et al. 2018;Albrecht et al. 2013;Amer et al. 2017;Birt et al. 2017a;Cowling and Birt 2018;De Oliveira et al. 2013;John et al. 2015;Küçük, Kapakin, and Göktaş 2016). There was a general acceptance of mMR as a learning tool and recommendation for further use in courses (Amer et al. 2017;Birt et al. 2017a;Cowling and Birt 2018). Usability surveys highlighted that mMR demonstrated a statistically significant decrease in fatigue, numbness and boredom compared to control groups (Albrecht et al. 2013;Moro et al. 2017) and lowered the cognitive load of learning for students (Küçük, Kapakin, and Göktaş 2016). Students also favoured access to video content of skill demonstration when AR markers linked to external sources (Garrett, Jackson, and Wilson 2015).

Constraint of mMR
Some studies found no statistically significant difference in skill competency between experimental and control groups (Bartlett et al. 2017;Birt et al. 2017b;Noll et al. 2014;Noll et al. 2017), nor when comparing AV (Oculus), AR (3D objects on tablet) and tablet-based tuition without AR (Moro et al. 2017). Noll et al. (2017) reported some 'irritability' in the use of the mARble platform, and Garrett, Jackson, and Wilson (2015) commented that technical issues, slow response time, phone incompatibility, Internet connection issues and instability of the platform negatively influenced students' perception of mMR. Students did not value AR marker links to text or PDFs and found the use of scanning markers disrupted the flow of learning.

Quality
While this review included a range of study designs, the majority were either one-off case studies; did not specify mMR exposure time; or exposure time was not reflective of that in a tutorial or self-directed learning -seeming to negate the premise that access to mMR should enable flexibility in student-directed learning (Bujak et al. 2013). However, some studies did encourage flexibility in student learning by offering the exposure to mMR throughout the course (Birt et al. 2017a(Birt et al. , 2017bCowling and Birt 2018;Küçük, Kapakin, and Göktaş 2016) or time similar to that in a 60 min tutorial (Bartlett et al. 2017). In an effort to embrace pedagogy before technology, Cowling and Birt (2018) employed the DBR methodology to analyse the 11 (page number not for citation purpose) problem, develop a solution in an initial phase, then test and refine in a second phase ( McKenney and Reeves 2012). Solutions to practical and complex educational problems are iteratively developed and impact on real-world problems in a framework for experimental yet rigorous inquiry (Cook and Santos 2016;McKenney and Reeves 2012). Design-based research may better enable the simultaneous pursuit of pedagogy and practical innovation in the emerging field of mMR.
Two studies (Noll et al. 2014(Noll et al. , 2017 included a follow-up period at 14 and 20 days, respectively. However, longitudinal studies across a semester would better determine the impact of mMR in retention of knowledge. Furthermore, as included studies were techno-centric with only one study involving a focus group (Garrett, Jackson, and Wilson 2015), future studies would benefit with interview-based data to understand the social process of learning.

Participants
The use of mMR in TEL was restricted to four health disciplines. However, the interventions included could be translated to use with physiotherapy, optometry, midwifery, oral health or occupational therapy students. Garrett, Jackson and Wilson (2015) integrated AR markers into a nursing clinical scenario requiring some decision-making regarding a patient in respiratory failure. A recent systematic review by Carvalho et al. (2017) highlighted that problem-based learning was the most used strategy in undergraduate nursing programmes to facilitate clinical reasoning -a crucial process in enabling autonomous healthcare practice. The use of mMR to develop similar scenarios could be utilised when designing for authentic learning and furthering high-order analysis.

Intervention
The TouchSurgery application was the most investigated form of mMR in this review with studies aimed at establishing validity with surgical modules. The Aurasma, Layar and mARble platforms were also utilised. These established platforms make development, delivery and marketing cost-effective compared to employing external developers to quantify mMR usability. Augmented reality procedural skills may also be less timeconsuming to develop compared to the development of AV environments with multiple options in a complex case scenario. Some studies adopted a bring-your-own-device (BYOD) approach, especially those using the TouchSurgery application. While tethered AV devices are commercially available, they are not considered mobile and affordable for the general higher education population at the time of this review. One included study compared AR-based anatomy on a tablet to AV-based anatomy using an Oculus Rift (Moro et al. 2017). This study found no difference in mean anatomy knowledge score, learner immersion and engagement. However, the AV participants exhibited more symptoms (e.g. headaches, dizziness, nausea, blurred vision) than the less immersive AR group.
The included studies did not engage with higher order critical analysis or take advantage of interactions that link social media with mMR (Crompton 2013). This may be a result of current software limitations and accessibility in the included mobile devices. Recently, AV using tethered HDUs has been used to investigate behaviours of general practitioners and trainees when confronted with a patient demanding antibiotics prescription (Nifakos and Zary 2014;Pan et al. 2016 present a real-life scenario and that experienced GPs are more likely to use clinical reasoning and rationalise withstanding the pressure to prescribe antibiotics unnecessarily. mMR problem-based TEL could be utilised in similar behavioural scenarios. There was an absence of using mMR in interprofessional collaboration, which may prompt future integrated practice, understanding of discipline-specific roles, development of common language and potentially overcoming barriers of distance between course venues (Cochrane et al. 2017). Future studies may also include interprofessional students interacting with each within the mMR scenario, as well as less linear procedural learning in an effort to enhance clinical reasoning and engagementwhich was only investigated by Moro et al. (2017) in the included studies.

Limitations of the review
The limitations of any systematic review are the scope of the search terms and the databases searched. The search was also restricted to papers published in the English language, resulting in studies from Australia, Canada, England, Germany, Turkey and the United States only. The age of the participants in the included studies was limited to a younger (<25 years) university population when indicated. Future inclusion of other disciplines (e.g. midwifery and occupational therapy) may reflect an age differential and perception of use of mMR in HHE.

Conclusions
The majority of studies in this review demonstrated a positive impact to the development of skill competency and knowledge by utilising mMR in HHE. While the included studies focused on four health disciplines, there is opportunity to extend mMR using available BYOD devices, developed software and skills-based measures to other areas of health education (see Table 2). However, there is a demonstrable lack of longitudinal studies and a general lack of engagement with learning theory in the design of the studies, interventions and evaluation methods. Future studies in mMR would benefit from utilising DBR, which enables inquiry into the transformative possibilities in this emerging field of mMR in HHE. The identified key drivers for the studies are clinical skill competency and flexible delivery for students or in response to limited clinical simulation facilities. There is also little engagement with higher-order critical analysis or capacity building for non-specific scenarios. This stems from a gap in embedding the studies in learning theory and higher-order critical thinking, which presents a challenge to future studies in HHE.