INTRODUCTION
Medical students and residents are increasingly challenged to learn more medical content at the same time as their predecessors.1 Furthermore, work hour restrictions can result in fewer opportunities for surgical training, further exacerbated by the COVID-19 pandemic, where many orthopaedic residency training programs faced a significant decrease in orthopaedic procedures.2 Meanwhile, orthopaedic surgical faculty at academic medical centers face increased pressure to be as clinically productive as possible. This can make finding time to teach in the conference room or workshops and labs harder. We, therefore, have a growing problem of too much to learn with fewer clinical and didactic teaching opportunities.
The online space provides opportunities beyond traditional learning methods. Webinars, lectures, podcasts, quizzes, and discussion boards are just a few examples of passive and active online learning opportunities. Virtual reality (VR) technology can potentially develop technical skills, allowing residents to practice safely and at their own pace. Some institutions have implemented a blended approach with a combination of virtual and in-person education.2,3 Multiple reports have assessed the implementation of online education in orthopaedic education. This review aims to provide an update on the current online education methods in orthopaedic surgery, with particular attention to the evidence for its support.
REVIEW
Blended Learning
Although there is no universal definition of blended learning, it is a combination of unstructured and structured learning, with educational material disseminated online to students via webinars, techniques videos, podcasts, PowerPoint presentations, etc., within a designated time frame before a formal in-person activity.4 With a blended learning approach lectures can be prerecorded and provided online, leaving time in conference for more active learning activities such as case discussions and workshops.
We identified three blended learning studies in orthopaedic surgery, including 300 participants (medical students, residents, fellows, and attendings). Study outcome measures are summarized in [Table 1].4–6
Senkoylu et al. developed a blended learning course for basic pediatric spinal deformity training, where they recruited 33 orthopaedic surgeons and neurosurgeons from the Turkish Spine Society.4 The online component consisted of a three-week period that provided course material in various online modalities such as webinars, operation videos, and online journals related to the learning outcomes the investigators developed, followed by the face-to-face (F2F) component, which consisted of case-based discussions on the learning outcomes. Participants were required to take a pre-quiz before instruction, one after the online learning component, and another after completion of F2F to assess the course’s efficacy. Only 13 of the 33 participants completed the F2F component of the system. Pre-course scores were recorded as 5.9 with a standard deviation (SD) of 2.3, before F2F, 6.8 with an SD of 1.6, and post-corrective scores were 7.9 with an SD of 0.8 (p =0.004).4 These results indicate that blended learning approaches might present an effective alternative to traditional learning methods.
Suwannaphist et al. investigated 252 5th-year medical students (128 in traditional and 124 in blended learning cohort) participating in their orthopaedic rotation in southeast Asia.5 They compared their traditional cohort from 2019 – 2020 (before the Corona Virus Disease COVID-19 pandemic) to a blended learning cohort from 2020 – 2021. Their blended learning course comprised a one-week online course containing webinars, imaging interpretation, online case-based discussion, and three weeks of inpatient and outpatient teaching in clinics and training in the operating room. Summative assessments included multiple choice quizzes, key features, modified essay questions, oral exams, and the objective structures of clinical examination, and calculated the grade point averages between the two cohorts. The student’s grade point averages were 3.2 and 3.3 in 2019–2020 and 2020–2021, respectively (p = 0.06).5
Chen et al. utilized WeChat, a phone-based instant message platform, as a blended learning platform for fourth-year medical students rotating in pediatric orthopaedics, where educators provided articles and questions relevant to the topic for independent study, followed by online case-based discussion, problem-based learning, and didactic lectures held the subsequent day.6 The study recruited 22 students to participate in the WeChat Pedagogy mode. It evaluated them in six domains of performance: possessing professional accomplishment, gaining knowledge, improving clinical skills, developing independent clinical thinking, promoting interpersonal skills and self-improving capability, and using the same evaluation tool; a maximum score of ten points was given for each item on a 60-point scale on the rotation exam. Exam scores were compared to traditional instruction cohort n=23 with WeChat Blended pedagogy mode, and traditional instruction scores were 47.27 and 44.52 (p=0.08), respectively. These limited preliminary studies suggest that the blended learning mode might be as effective as conventional methods, although these were not explicitly designed to prove non-inferiority.6
Utilizing blended learning presents several advantages, such as students’ ability to access learning tools at times that most benefit their schedule. Furthermore, institutions would no longer rely primarily on their faculty to instruct students, as several professional societies have developed webinar series on topics easily accessible to students and educators nationwide.2 However, blended learning also presents challenges, such as the need for more interactions and social support between educators during the knowledge acquisition phase. Additionally, educators may need help recognizing topics students may find challenging.6 Implementing active evaluation tools during the online knowledge acquisition phase is imperative with consistent feedback mechanisms in place. Overall, blended learning provides a feasible alternative and intriguing option compared to traditional learning methods by emphasizing utilizing the modern advantages of technology.
Orthopaedic In-Training Exam (OITE) Preparation
Orthobullets (OB)(https://www.orthobullets.com), American Association of Orthopaedic Surgeons (AAOS) ResStudy (RS)(https://www.aaos.org/education/examinations/ResStudy), and the Journal of Bone and Joint Surgery Clinical Classroom (CC)(https://clinicalclassroom.jbjs.org) are three of the most popular online Orthopaedic In-Training Exam (OITE) study resources, with OB being the most preferred among residents.7,8
Orthobullets
OB provides free educational content in a multimodal fashion through case-based learning, question banks, videos, techniques, guides, imaging, discussion forums, and flashcards. They also offer their paid PASS Curriculum, which offers additional questions and tracks user progress. OB question banks are not derived from previous OITE as is with the RS; however, the authors crafted questions to mimic the OITE style. OB can retain some functionality without an active internet connection when used with their downloadable app.
ResStudy
RS provides OITE-style questions derived from previous OITE, which come with detailed explanations and access to the established literature but lack educational content features like videos, articles, flashcards, and spaced repetition algorithms that OB and CC possess. RS requires an active internet connection and subscription for content access.
Clinical Classroom
CC utilizes multiple learning tools: multiple-choice questions, fill-in-the-blank questions, and flashcards, and incorporates spaced repetition algorithms to tailor the learning experience based on the user input on the perceived difficulty of different contents. This learning style has been proven effective in other modalities.9 However, their questions are derived from experts not designed in an OITE format, unlike OB and RS. CC requires an active internet connection and subscription for content access.
Comparative Analysis of OITE Platforms
CC ranked the lowest when evaluating the differences in knowledge acquisition and preferred resources.7 Volaski et al. found that OB was rated higher amongst all surveyed for fact acquisition, relevance for OITE success, answer explanations, and contribution to a knowledge base. However, when surgeons and senior residents were analyzed separately, CC was rated superior to OB for its impact on clinical practice and ability to address clinical understanding in varied answer fields rather than simple multiple choice. Though CC was rated higher in this advanced cohort, the difference was not significant by statistical testing. This study highlights potential nuanced differences in where and how these platforms focus on specific knowledge and how training level might influence the perspective of clinical knowledge/importance.10 We identified three studies that investigated the efficacy of these OITE prep platforms, which included 517 participants, as summarized in [Table 2].7,8,11
Boody et al. studied the efficacy of the OB PASS Curriculum as a preparation resource for the OITE. The pilot study included residents from five United States (US) orthopaedic residents with 71 residents. Residents were required to complete the PASS Curriculum monthly quizzes for assessment analysis, and program directors from the residency programs submitted their 2013 and 2014 OITE exam scores.7 Overall, there was a 2.5% increase in OITE score, increasing on average from 163 to 176 (p=0.406). However, the sub-group analysis revealed that junior residents PGY 1 and 2 (n=28) demonstrated significant increases from 139 to 164, an average 10.1 % increase (p=0.0259).7 The results indicate that OB is efficacious in acquiring knowledge for residents, especially junior residents.
Theismann et al. assessed residents’ preferred study source and its impact on their OITE scores. They determined that 82% of 386 residents recruited in the study use online platforms, including OB, CC, and RS, with OB reported as the most utilized source.8 The study then compared the 2018 and 2019 scores of 283 residents from nine US residency programs. All preferred study methods demonstrated an increase in OITE scores, with OB showing an average rise of 2.6 percentile points n=139, RS with 3.4 percentile points n=80, and CC with seven percentile points. Additionally, there were no statistically significant differences in OITE scores between students who preferred online platforms and those who chose paper study sources, and there were no statistically significant differences in OITE scores between selected online resource users.8
Margalit et al. also compared OB, RS, and CC efficacy in OITE preparation. The investigators recruited sixty residents from US allopathic medical programs to participate in the study, where they were expected to complete a 15-day course subdivided into three five-day blocks consisting of trauma, pediatrics, and joints, where residents will rotate the online platform each block.11 A pre-and post-test assessed the knowledge gained within each topic. Across all three platforms, there was a 4.4% increase in scores on average. OB demonstrated the most improvement when analyzing each platform individually, OB (6.8%), RS (5.4%), and CC (1.0%). When comparing score improvements respective to each platform against each other, OB’s improvement against CC was statistically significant (p<0.0018).11
Based on the aforementioned results, all three platforms effectively acquire knowledge for the OITE. OB is the most utilized resource, the most readily accessible, and the one with the most improvements in performance. However, these conclusions are limited, and further studies are needed to analyze these platforms further to assist students and residents with knowledge, especially since these platforms undergo constant changes. Lastly, all learners acquire knowledge differently, so personal preference is a contributing factor.
Virtual Journals Clubs
Traditionally, attendings, residents, and medical students meet in person to conduct periodic journal clubs to review approximately two to six articles critically. Journal clubs are spaces to learn about new medical developments and keep medical providers updated on current standard practices in care. However, a limitation of this is the difficulty in coordinating schedules to hold journal clubs. Some have explored social media platforms like Twitter and Reddit as alternatives to mitigate this issue, as participants can engage in discussion when it suits their schedule. However, a lack of structure, low engagement, and limited ability to restrict access to only virtual club members are adequate concerns for utilizing social media as a medium for journal clubs. Therefore, instant messaging applications like WhatsApp and Viber show promise to be effective mediums for journal clubs due to their ability to address such concerns.12,13 We identified two studies that investigated the efficacy of virtual journal clubs.
Peponis et al. assessed the Viber app as a modality for a virtual journal club. The study included 73 users, 68 from Ethiopia and five from the United Kingdom (UK). Seven articles were analyzed, and 60 quiz questions were asked. The satisfaction survey had 20 responses, resulting in 85% reporting that the virtual journal club improved their research skills. 95% of responses reported that Viber Journal Club was superior to traditional ones.12
Clesham et al. examined WhatsApp’s efficacy as a platform for virtual journal clubs. For six weeks, a moderator selected one article for discussion to provide ample time for analysis. Before and after the study, 28 multiple-question questions assessed participants’ knowledge of research methodology. Orthopaedic resident average scores improved from 48.2 to 69%.13
Critical features of influential journal clubs include scheduled meetings with mandatory attendance, rotating faculty/student-led discussions, evaluation tools, and summary points.5,12,13 However, as mentioned previously, coordinating schedules can be challenging. To bypass these obstacles, the investigators in Peponis et al. and Clesham et al. designed a similar study where a discussion lead selected an article for review within a designated period and then provided relevant questions in a staggered structure to facilitate critical appraisal of the article. At the end of the discussion week, discussion leads provided summary points. The researchers evaluated participant thoughts in each respective study and reported similar sentiments regarding improvements in critical appraisal skills, research methodology, and time dedicated to article review.12,13 Improvements to these findings could be attributed to the asynchronous portion of the article review, where participants participate in discussions based on their availability.
Virtual Reality
Virtual reality (VR) simulates surgical procedures using 3D technology and haptic devices. Such technology provides advantages in surgical training by creating an immersive environment where trainees are free to make mistakes. The technology is more accessible than cadaveric models and cheaper than simulators. It is most beneficial to junior residents who often assist rather than act as primary surgeons in actual cases.14,15 We reviewed three VR studies, including 69 participants, summarized in [Table 3].14–16
Luca et al. designed a study exploring VR in spine surgery training. The investigators recruited ten participants, three were senior spine surgeons, and seven were orthopaedic residents and junior orthopaedic surgeons. The first group, containing senior surgeons, had yet to experience VR simulations, which were used to determine familiarity with the technology. The participants were instructed to perform two virtual trials of lateral spine access. They were assessed based on the number of errors committed during preoperative and intraoperative simulations in each test. The study revealed that errors in preoperative settings decreased from 5.2 to 1.8. During the intraoperative phase, the average number of errors decreased from 4 to 1.4.16 These results indicate that simulations may reduce the learning curve when learning surgical techniques.
Huri et al. designed a study to compare the efficacy of simulation-based shoulder arthroscopy training to cadaver-based training. The investigators recruited 34 residents and divided them into a cadaver or simulation group. They were later assessed using a VR simulator and scored using a talent test to assess knowledge gain. The talent test comprises two parts: diagnostic and therapeutic. In the diagnostic segment, participants identify anatomical landmarks and mark any pathologies within the shoulder joint. The therapeutic part involves removing loose bodies from the shoulder within five minutes. Various metrics, such as procedure time, complication rates, and path lengths, are recorded digitally. Three scores are calculated for each participant: total score, safety score (reflecting complications), and economy score (indicating efficient tool usage).14 However, individuals in the cadaver group needed more time to complete arthroscopy sessions compared to the simulation group. Additionally, the cadaver group scratched the humeral head cartilage more often than the simulation group (p=0.0044).14 One significant confounding variable is VR usage, where participants in the VR group used the same program to assess knowledge of improvement. Another disadvantage of using VR seems to be that haptic control devices may not correctly emulate real-world scenarios. Additionally, most VR simulations are only limited to arthroscopy. More programs are needed to simulate all the various procedures orthopaedic surgeons perform.
Orland et al. designed a randomized controlled study comparing the efficacy of VR with or without technique guides compared to technique guides alone in intramedullary tibial nail insertions. The investigators recruited 25 first and second-year medical students. They divided them into three groups: nine students in the VR and technical guide groups, eight students in the VR group without the technical guide, and eight in the technical guide group alone. The VR groups were required to participate in three separate sessions, with the first requiring at least a minimum of three trials to reach 85% accuracy and the subsequent two requiring at least a minimum of two attempts to get 90% and 95%, respectively. Then, all three groups performed the procedure on an artificial compact bone. Overall, the VR groups with or without the technical guide completed the procedure at a higher proportion than the technical guide groups alone (p=0.001). Additionally, the VR group committed fewer errors than the technical guide group alone (p=0.02). There were no statistical differences between the VR groups with or without technical guides.15
Although this study showed promising results, it is imperative to acknowledge its limitations. The study had a small sample size. Additionally, the investigators required the VR groups to earn an 85% to 95% accuracy rate before participating in the compact bone simulation, where the investigators did not impose such a requirement on the technical guide group.
Podcasts
Audio podcast material allows learners to listen to educational information. They can often be heard while multitasking, efficiently using time. With limited time during the day available for medical practitioners, they can learn while commuting or doing other daily tasks. The information presented can range from journal reviews, interviews with experts, or summaries on high-yield topics in orthopaedics.
Back et al. designed a randomized control trial (RCT) to compare learning outcomes of students learning orthopaedics via text or podcast. Participants were given 20 minutes with their learning modality to learn a specific topic in orthopaedics and then assessed after the learning period. Each student took a pre and post-test to determine the efficacy of each modality in the text and podcast group. In the text group, the pre and post-test scores were 48.1% and 60.9%, and the podcast scores were 44.5% and 66.2%. Both groups revealed a significant gain of knowledge (p<0.001), and podcast groups demonstrated a significantly higher increase in knowledge compared to text groups (p<0.021).17
Back et al. designed a follow-up study in an uncontrolled study (UCS) to evaluate the learning outcomes of both modalities that are more akin to a real-world scenario where there were no restrictions on how each group modality was utilized. The findings were similar to the RCT, where the podcast group demonstrated a higher knowledge gain than the written text group. The investigators also determined no significant differences between the results of the UCS and RCT (p=0.396). The podcast group demonstrated a higher knowledge gain than the text (p<0.011).18
One interesting finding in these studies is that participants reported a higher satisfaction rate with digital text modalities in the UCS than with RCT (p<.005). This is due to the lack of time restrictions implemented in text vs. podcast utilization in the UCS compared to the RCT. Participants were free to utilize digital text based on their apparent needs.17,18
CONCLUSION
Although online learning cannot entirely replace in-person education and training in orthopaedic surgery, it can be incorporated into medical education and postgraduate residency training in numerous ways to supplement in-person learning and create more time for active learning activities in conferences. Although few studies directly and objectively compare the two, it is worth pursuing due to its unique advantages in modern medicine and life.
DECLARATION OF CONFLICT OF INTEREST
The authors do NOT have any potential conflicts of interest related to the content presented in this manuscript.
DECLARATION OF FUNDING
The authors received NO financial support for the preparation, research, authorship, and publication of this manuscript.
DECLARATION OF ETHICAL APPROVAL
Institutional Review Board approval was not required for the production of this manuscript.
DECLARATION OF INFORMED CONSENT
There is no information (names, initials, hospital identification numbers, or photographs/images) in the submitted manuscript that can be used to identify any patients.
ACKNOWLEDGEMENT
We would like to acknowledge Jenny Pierce, Lewis Katz School of Medicine librarian, for her assistance with the literature review and acknowledge the Temple University Hospital Department of Orthopaedic Surgery and Sports Medicine for general support of this project.