Introduction
Medical education and surgical training are constantly evolving to be more efficient and cost effective.1,2 Work hour restrictions and increased performance expectations have left medical students and resident physicians needing alternative methods to practice surgical techniques.3 Technological advancements in the past decade have led to the development of surgical simulators, enabling trainees to practice surgeries outside direct patient contact. Current simulation modalities range from simple designs with basic hardware materials to robotic simulators and, more recently, virtual reality and smart device-based applications.4,5 One such smart device-based application (app) is Touch Surgery (London, England, UK). This interactive multimedia application comprises hundreds of step-by-step modules for surgeries and procedures in various specialties. Users interact with the app in multiple ways, from swiping to simulate surgical incisions, tapping for sutures, or answering multiple-choice questions regarding what tool to use next. This app teaches surgical steps rather than surgical psychomotor skills. Surgical step knowledge includes knowing the relevant anatomy, the steps of the surgery in the correct order, and when to use the proper instruments. In contrast, psychomotor skills refer to physically handling tools and tissue.6 This acquisition of surgical step knowledge parallels traditional teaching styles, such as lectures or preoperative textbook preparation. In contrast, psychomotor operative skills have historically been taught using an apprenticeship model whereby a more advanced surgeon provides real-time, direct feedback to trainees.7
Previous studies have shown that E-learning, or mobile digital education, is as or more effective than traditional learning modalities for acquiring knowledge.2 Concurrently, surgical performance is a commonly assessed entity with numerous validated rating scales existing to accurately assess skill and level of mastery, mainly as they apply to orthopaedic hand surgery.1,8–10 While limited research has suggested that mobile-based surgical simulators like Touch Surgery can improve the initial performance of real-time surgeries or procedures, no studies have directly assessed the interplay between surgical simulation modules and first-time surgical performance, nor have they included an in-person trained cohort reflecting an apprenticeship model of surgical teaching.11 Furthermore, the real-life translation of surgical simulation learning has yet to be studied in orthopaedics despite the emergence of virtual reality learning tools in recent years.5,12
The primary aim of this study was to compare the effectiveness of different surgical training tools when translating procedural attainment to physical surgical performance, using trigger finger release (TFR) as the surgery tested. The study’s secondary aim was to evaluate participants’ satisfaction and perceived usefulness of the different education modalities. It was hypothesized that those using a mobile device-based surgical simulator by way of Touch Surgery would have better initial surgical performance scores and rate the utility and enjoyability of their modality the highest.
Materials and Methods
Database and Study Participants
Institutional Review Board approval was first obtained. This study was approved by the Thomas Jefferson University Institutional Review Board on September 7, 2021; Control #: 21E.744.
Subsequently, all first- and second-year medical students at Sidney Kimmel Medical College received an interest survey via email, recruiting them to participate in this study. Out of those who expressed interest, students were excluded if they had previously used the Touch Surgery app, had hands-on surgical experience performing a TFR surgery, or had observed a TFR surgery. From those who expressed interest, a random list generator selected 27 students to participate. Twenty-seven was the number of students selected, as this was the number of cadaver arms available to perform the study’s surgical portion. From there, the 27 students were randomized into three cohorts of nine students via an online tool, RandomLists.com. Each cohort was assigned one learning modality with which to learn the TFR surgical steps: surgical simulation through the Touch Surgery app (cohort “SS”), video lecture sourced from VuMedi (Oakland, CA, USA) (cohort “VIDEO”), or through live teaching by a board-certified orthopaedic hand surgeon on a cadaver arm (cohort “LIVE”). To ensure a similar operating interface, students in the SS and VIDEO cohorts completed their modalities using standard iPad tablets (9.4 inches × 6.6 inches; Apple, Cupertino, CA, USA). The content provided in the SS, VIDEO, and LIVE teaching sessions was uniform among modalities; the content in SS and VIDEO were identical, with the LIVE teaching script created and validated by an orthopaedic hand surgeon to reflect the other cohorts’ content. Each cohort reviewed the TFR surgery using their assigned modality three times consecutively in one sitting for repetition and familiarity.
Outcome Measures
All students then completed a standardized 16-question multiple choice test evaluating their knowledge of the procedural steps [Appendix A]. This test asked participants questions regarding the next steps of the surgery, tool usage, and techniques. The assessment questions were evaluated by hand surgery fellowship-trained board-certified orthopaedic surgeons prior to administration. Following this, students attempted to perform the TFR surgery on a finger from a cadaver arm using the tools and steps outlined in their learning modalities [Figure 1]. Fresh frozen and thawed cadaver arms were supplied by Science Care (Phoenix, AZ, USA). A board-certified orthopaedic hand surgeon evaluated student performance using a detailed TFR checklist, a GRS, and a modified objective structured assessment of technical skills (OSATs) pass/fail assessment [Appendix B]. The surgeon was blinded to the student’s respective cohort. These tools have been previously validated as appropriate measures of TFR surgical performance.10 The TFR detailed checklist consisted of each primary step of the surgery listed in consecutive order, so each step could be marked as “yes” for done correctly or “no” for not done or done incorrectly by the evaluator as the participant performed the surgery. The GRS evaluated participants on soft surgical skills such as respect for cadaveric tissue and not causing unnecessary damage, tool handling, and operation flow. The OSAT’s pass/fail assessment took participants’ overall performance and any adverse events into account. Adverse events resulting in an automatic failure included laceration of the neurovascular bundle and lack of tendon transection. The time each participant took to complete the surgery was also recorded to gauge participant efficiency.
Prior to utilizing their learning modality, students filled out a confidence questionnaire assessing their confidence in their ability to learn the steps of and perform the TFR surgery. They then filled out this questionnaire again post-modality pre-cadaveric performance and post-modality post-cadaveric performance to evaluate their confidence throughout the study [Appendix C]. All cohorts also completed a post-study questionnaire assessing their assigned teaching modality’s usefulness, effectiveness, and enjoyability [Appendix D]. All surveys were on a 5-point Likert scale assessing agreeableness to predetermined statements in which each score was defined as follows: 1. Strongly disagree; 2. Disagree; 3. Neither agree nor disagree; 4. Agree, and 5; Strongly agree.
Statistical analysis
Parametric continuous data is presented as mean (SD), and p values were calculated by performing an ANOVA test. This includes the analysis of TFR assessment scores and evaluations of cadaveric performances, as well as the confidence and post-modality questionnaires. Categorical data is presented as cell count (%). Chi-Square or Fisher’s Exact tests were used to calculate p values for categorical data. This includes the analysis of the demographics of participants. P values less than 0.05 were deemed significant. Multiple comparison testing was used on significant values when comparing three groups. All statistical analyses used R Studio (Version 3.6.3, Vienna, Austria).
Results
Participant Characteristics
Twenty-six of the 27 medical students meeting the inclusion criteria and enrolled completed the entire study, including their assigned learning modality, all the questionnaires, assessments, and cadaveric surgery. One first-year student in the VIDEO cohort did not perform the surgery on a cadaver, complete the post-cadaveric confidence questionnaire, or evaluate the modality’s usefulness due to needing to leave the study early. Of the study participants, 85.2% were first-year medical students, with 14.8% being second-year medical students. Two-thirds (66%) of participants were male and one-third (33%) were female [Table 1].
Assessment Scores
There were no statistically significant differences between the results of each cohort on the TFR assessment (P=0.123) or cadaver performance as graded by the steeps checklist (P=0.549), GRS of soft surgical skills (P=0.567), OSATs pass/fail (P=0.874), and time to surgery completion (P=0.711) [Table 2, Figures 2A-D].
Confidence Questionnaire Results
Regarding the pre-modality pre-cadaveric, post-modality pre-cadaveric, and post-modality post-cadaveric confidence questionnaires, there was no statistically significant difference in responses between groups on any of the surveys [Table 3].
Evaluating Modalities
When evaluating modality usefulness, effectiveness, and enjoyability, students in the LIVE cohort rated their modality as an overall better training tool, particularly when rating its usefulness as a training tool (P=0.041), assessment tool (P=0.006) and their first-choice method of surgical preparation (P=0.009) When evaluating the modalities on whether they were easy to use, follow, or understand, the SS group rated their modality the highest with statistical significance (P=0.010). All other components of the modality questionnaire did not result in statistically significant scores between groups [Table 4].
Discussion
The results of this study did not support the hypothesis. Still, they demonstrated that the modality used to learn surgical steps has no implications on the first-time surgical performance of medical students as graded by the yes/no step checklist, GRS of surgical skills, and OSATs pass/fail assessment. Furthermore, these results did not show that any modality is superior for surgical step knowledge attainment as evaluated by the TFR multiple-choice test. Regarding secondary objectives involving the analysis of modality usefulness and enjoyability, live teaching was rated the highest in usefulness as a training tool, usefulness as an assessment tool, and as trainees’ first-choice method of surgical preparation. The surgical simulator was rated the easiest to use, follow, and understand. There were no differences between cohorts when evaluating their confidence in performing the TFR surgery at any point during the study.
Previous studies have demonstrated that mobile digital learning tools are superior to traditional forms of learning for procedural step acquisition.2 Specifically, the Touch Surgery app has been assessed, validated, and proven to improve student learning of the steps of numerous surgeries.13–16 In the field of orthopaedics, one study has previously published on the utility of Touch Surgery to teach medical students the steps of carpal tunnel release surgery, demonstrating enhanced procedural step assessment performance in the Touch Surgery group than a control group learning from a traditional slide deck with audio overlay.13 While this study did not reproduce prior findings with statistical significance, our assessment scores do not support SS being inferior to traditional methods and, in fact, favor SS being superior to other forms of learning with the highest average test score compared to the video and live teaching groups.
Throughout our research, few studies sought to evaluate how the method used to learn surgical steps affects surgical performance. Module-to-skill-based studies have historically centered around communication or physical exam skills, for which the results have positively supported using mobile-based apps, audio simulations, or the like.2,17,18 Surgical simulators have traditionally been studied regarding their ability to enhance psychomotor skills, striving for fidelity to actual surgery. For example, arthroscopic simulators are widely used in orthopaedic surgical training and have consistently enhanced actual surgical performance.19 These simulators are most useful once a learner has already mastered procedural steps. It has been previously published that knowing the procedural steps of a surgery opens “attention bandwidth” for trainees to focus on psychomotor and soft surgical skills when using a simulator or working in the OR.6 Given this, the need was identified to study the relationship between a low-fidelity simulator, like Touch Surgery, against traditional forms of procedural step attainment in preparing one for first-time psychomotor performance.
The studies that have specifically investigated methods of step acquisition as they relate to surgical, or similarly procedural, performance have demonstrated mixed results. A study by Haubruck et al. demonstrated that medical students who completed the Touch Surgery chest tube insertion (CTI) module performed significantly better on a porcine model CTI simulation than those who completed a control module following a lecture on CTI.11 While Haubruck et al.'s study strongly parallels this one in the overall design and the use of medical students as participants, there is a steeper learning curve with surgical techniques and tool handling in a TFR surgery than CTI, which may account for the difference in findings. The TFR surgery requires knowing how to use numerous tools to handle delicate tissue and dissect appropriate structures. In contrast, CTI is a less precise and delicate procedure with fewer steps focused on getting a tube into the chest cavity. These results suggest that simulation-based modalities in new learners may be best suited for simple, step-by-step procedures.
Supporting this theory, a study by Bunogerane et al. showed that surgical residents who used Touch Surgery for learning tendon repair surgical technique versus a textbook scored significantly higher on tendon repair simulation and reported high application satisfaction.14 Bunogerane et al.'s study found a statistically significant difference in SS participants who were surgical residents with baseline surgical skills prior to the trial, indicating that prior surgical experience does affect the translation of procedural knowledge to surgical performance. The participants in this study had no surgical training and, therefore, had an added obstacle to overcome.
This study is similar to the work by Nilsson et al. that looked at the effectiveness of mobile app-guided against textbook-guided training in ultrasound-naïve residents of extended focused assessment with sonography for trauma (eFAST) exams. Their study demonstrated no difference in test performance or diagnostic accuracy between the two groups.20 Just like the participants in this study were new to both the steps and surgical skills needed to perform a TFR surgery, the residents in that study were both technically and diagnostically ultrasound naïve. They were novices at using the ultrasound machine and interpreting the images. Therefore, their skills may have been too rudimentary to demonstrate the benefit of different forms of initial training.
The secondary outcomes of this study looked at user experience with each learning modality. Those in the LIVE cohort rated it higher compared to the SS and VIDEO groups as the most useful training and assessment tool, as well as their overall first-choice way to learn surgical steps. This cohort most accurately depicts the apprenticeship teaching model of surgery, which, to our knowledge, has not been studied against newer SS technologies to learn surgical steps. The apprenticeship model has persisted in surgical training since the 18th century, whereby surgery is taught by the trainee directly observing and imitating the actions of a more experienced trainer. This model requires opportunity for repetition through high case volume, long work hours to provide adequate training time, and skilled surgical teachers.7 Given the resident work hour restrictions now in place, sub-specialization of surgeons, and increased number of residents per training program, there has been a need to shift this paradigm to include more SS in surgical training. This study supports previously published work documenting the validity of SS in medical education, with the SS group rating the modality as easiest to use, follow, and understand.2,13–16
However, while SS has shown utility in surgical education, the importance of mentor-mentee relationships cannot go unrecognized. Numerous studies have cited mentorship as the most important factor in medical students pursuing surgery and surgical residents picking a surgical subspecialty.21–24 Mentorship is born from personal interaction between an educator and learner, which was only accessible to those students in the LIVE cohort. The results show that live teaching was the first choice among students to learn surgical steps, therefore parallels these ideals, supporting the utility of interpersonal interactions, mentors, and the historical apprenticeship model on career trajectory among trainees.
Overall, the results of this study support the use of SS in orthopaedic surgical training and show that SS is not inferior to more traditional methods of learning as it pertains to surgical step acquisition, translation of surgical knowledge to physical performance, and is an easy-to-understand tool. Simulation and E-learning resources provide alternative viable learning platforms whose accessibility does not fluctuate. This is particularly beneficial in areas of the world with limited teaching resources and in the current state of orthopaedic surgical education. Changes in work hour restrictions and the COVID-19 pandemic led to decreased case volume and operative time for residents and limitations in clinical rotations for medical students.3,25,26
Readily available educational modalities for surgical knowledge attainment have a greater impact on surgical training beyond their ability to provide out-of-hospital learning opportunities. Surgical training consists of both cognitive and psychomotor elements, meaning one needs to learn the steps, tools, and anatomy, as well as how to operate physically. It has been discussed that focused training in one area makes those skills more natural, giving the learner more time to focus on the other area.6,15 As such, using SS that promotes cognitive mastery outside of the operating room may allow for focused psychomotor training in the operating room, creating competent surgeons more efficiently.
We also evaluated medical students on their preferred instruction modality, which elucidated live teaching as their first choice of learning surgery. Virtual curriculum has apparent advantages, especially in environments that limit in-person instruction. However, identifying the learning style of choice is crucial. As orthopaedic surgeons, there is a dedication to lifelong learning. Therefore, identifying and selecting a preferred learning style has the potential to impact satisfaction as a learner throughout an entire career. While there is evidence that supports that one learns a skill no better if it’s in their preferred learning style or not, the enjoyability and variety are essential of accessible learning tools throughout a lengthy training process.27,28
Strengths of this study include the addition of the LIVE cohort. All similar previous studies compared lecture-styled learning to simulation. Including a live teaching cohort more accurately reflects the current model of surgical education. Another strength of this study is using Touch Surgery as our SS model. Touch Surgery has been validated as a reliable SS education modality in numerous previously published studies.13–16 The most significant weakness of this study is the sample size. Including more medical students would have added strength to our data, power to this study, and insight into this topic. The interest level and baseline knowledge of each medical student are likely vastly different, which could have impacted their capacity to learn and perform this task. Furthermore, while the evaluations used for grading surgical skills have previously been well utilized and validated, they have primarily been studied using residents and attending surgeons with foundational surgical ability as participants.10 This study’s participants had no surgical ability, which may have limited the ability to accurately assess their learning from educational modality to surgical performance.
Areas of future research should include repeating this study to increase sample size and study power and investigating teaching modalities related to surgical training costs and burnout in medical students, residents, and attending education curriculum. In medicine, we are dedicated to continued education and should always strive to improve the ways we learn.
Conclusion
All educational modalities should be considered in orthopaedic surgical training. This study showed medical students perform cognitively and technically well when educated using virtual and live formats and that virtual formats are not inferior when live teaching is unavailable. However, they preferred live teaching. As such, a combination of virtual and live teaching in training programs is a supported model through the results of this study.
Declaration of conflict of interest
The authors do NOT have any potential conflicts of interest for this manuscript.
Declaration of funding
Financial support for the execution of the study and production of the manuscript was provided by The Rothman Institute at Thomas Jefferson University.
Declaration of ethical approval for study
This study was approved by the Thomas Jefferson University Institutional Review Board on September 7, 2021; Control #: 21E.744.
Declaration of informed consent
The authors declare there is no information in the submitted manuscript that can be used to identify patients.