MR MANIKIN

MR Manikin is a Mixed Reality (MR) critical care simulation project that creates a high fidelity simulation that can be dynamically adjusted with a focus on team interactions and clinical decision making. Critical care simulation relies on the ability to examine the patient and quickly make decisions based on the signs and symptoms the patient shows. For this simulation, we added visual cues such as bleeding and vomiting, auditory cues such as heart and lung sounds as well as critical reactions to examination such as the pupillary light reflex. All of these clinical signs and symptoms are dynamically controlled by the instructor through an iPad interface. In addition, the virtual environment has a vital signs monitor that is dynamically responsive to the instructor input and various imaging results can be displayed in the environment – again controlled by the instructor and made visible in response to student requests. The facilitator can send a situation to multiple students at the same time with a network connection using a control device.

With this, the students can practice and experience the decision-making process in critical care situations.
Tech: Unity | Microsoft Hololens 1 | Vuforia Image Tracking | Photon Unity Networking

Key Features

Immersive CPR Experience
   – check pupillary response, cardiac and lung sounds from the CPR mannequin just like a real-life patient
Control the Situation
   – control all the visual, auditory and biomedical signs on the mannequin
Critical Thinking Process
   – the mannequin will respond based on your decisions
   – stop thinking “what is next” and start thinking “what should I do”

OCS (Online Clinical Simulation)

Our COVID response – taking MR Manikin to a remote and online environment

We wanted to put together a simulation environment that would work for remote and online learning during COVID19. Key features of the MR Manikin experience are the team interaction and the dynamic live control by an instructor and we wanted to maintain this functionality. The clinical simulation group discussion is really important for medical students to learn clinical decision making and teamwork. We developed an app where students can view the critical care mannequin with augmented reality (AR) on their personal mobile devices and still be connected with the team through videoconferencing software. The entire experience is controlled by a facilitator: on their personal mobile devices students can check the ECG monitor, medical images, and body reaction based on scenarios presented by the facilitator. The facilitator can project a scenario to multiple students at the same time with a network connection and a control device. With this immersive encounter, students can experience and practice the decision-making processes involved in an emergency situation such as cardiac arrest. The main communication will be done through a 3rd-party video conference call platform like Zoom, Google Meet or Microsoft Teams.
Tech: Unity | Vuforia | Photon Unity Networking | Mobile Devices (alternate AR device to Hololens)

Key Features

Study From Home Experience
   – students and facilitator can run the simulation at home without in-person meetings
Team Training
   – students can simulate clinical practices in relation to their viewpoint and location of the MR manikin

Team

Dr. Kevin Shi
Dr. Claudia Krebs
Dr. Cynthia Liu

Sean Jeon
Ishan Dixit
Dante Cerron
Kimberly Burke
Silver Xu
Friend Prakitpong
Gabriel Chen
Arjun Vadehra
Degly Sebastian Pava
Lucy Lin
Jason Kim