KEYWORDS: Holography, Education and training, Mixed reality, 3D modeling, Solid modeling, Data modeling, Motion models, 3D displays, Surgery, Perceptual learning
Purpose: Virtual holographic simulation skills training has a growing role in supplementing surgical skills training given increasingly limited healthcare resources and recent advancements in mixed reality technology. However, creating highfidelity 3D holographic animations (particularly hand animations) is expensive, time-consuming, and complex. We propose a low-cost solution using mixed reality headsets for motion capture of hands to automatically create high-fidelity 3D holographic hand animations. Methods: In this study, a 3D animation of a single-handed knot tie was created using the Oculus Quest 2 and APS Mocap Fusion app for display on a Microsoft HoloLens 2. To assess the feasibility and quality of the created 3D holographic animation, a qualitative and quantitative pilot study of 20 participants was conducted comparing learning one-handed knot ties from an in-person demonstration versus the 3D holographic hand-tie animation. Results: Our pilot study demonstrated participants were able to learn one-handed knot ties from the holographic animation (70% of participants) and was comparable to in-person (80%). Promisingly, based on the Likert scale questionnaire, participants found learning from the holographic animation was more effective (4.4 vs 3.3), easier (3.6 vs 3.3), and felt more confident in learning the knot-tie (4.4 vs 3.5) in comparison to in-person demonstration. Furthermore, participants felt the holographic animation was comparable to real-life hands (4). Overall, we successfully illustrated a low-cost automated methodology of creating high-fidelity 3D holographic hand animations from mixed reality headset motion capture data with potential for use in surgical simulation skills training.
PURPOSE: Virtual reality and simulation training improve skill acquisition by allowing trainees the opportunity to deliberately practice procedures in a safe environment. The purpose of this study was to find if there was a difference in the amount of improvement the Perk Tutor, an augmented reality training tool, provided depending on the complexity of the procedure. METHODS: We conducted two sets of spinal procedure experiments with different levels of complexity with regards to instrument handling and mental reconstruction – the lumbar puncture and the facet joint injection. In both experiments subjects were randomized into two groups, Control or Perk Tutor. They were guided through a tutorial, given practice attempts with or without Perk Tutor, followed by testing without Perk Tutor augmentation. RESULTS: The Perk Tutor significantly improved trainee outcomes in the facet joint experiment, while the Perk Tutor and the control group performed comparably in the lumbar puncture experiment. CONCLUSION: Perk Tutor and other augmented training systems may be more beneficial for more complex skills that require mental reconstruction of 2-dimensional images or non-palpable anatomy.
PURPOSE: Gaining proficiency in technical skills involving specific hand motions is prevalent across all disciplines of medicine and particularly relevant in learning surgical skills such as knot tying. We propose a new form of self-directed learning where a pair of holographic hands is projected in front of the trainee using the Microsoft HoloLens and guides them through learning various basic hand motions relevant to surgery and medicine. This study looks at the feasibility and effectiveness of using holographic hands as a skills training modality for learning hand motions compared to the traditional methods of apprenticeship and video-based learning. METHODS: 9 participants were recruited and each learned 6 different hand motions from 3 different modalities (video, apprenticeship, HoloLens). Results of successful completion and feedback on effectiveness was obtained through a questionnaire. RESULTS: Participants had a considerable preference for learning from HoloLens and apprenticeship and a higher success rate of learning hand motions compared to video-based learning. Furthermore, learning with holographic hands was shown to be comparable to apprenticeship in terms of both effectiveness and success rate. However, more participants still selected apprenticeship as a preferred learning method compared to HoloLens. CONCLUSION: This initial pilot study shows promising results for using holographic hands as a new effective form of self-directed apprenticeship learning that can be applied to learning a wide variety of skills requiring hand motions in medicine. Work continues toward implementing this technology in knot tying and suture tutoring modules in our undergraduate medical curriculum.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.