NA-MIC Project Weeks
Back to Projects List
AR in Slicer
Key Investigators
- Alicia Pose Díez de la Lastra (Universidad Carlos III de Madrid, Madrid, Spain)
- Javier Pascau (Universidad Carlos III de Madrid, Madrid, Spain)
- Rafael Moreta Martínez (Universidad Carlos III de Madrid, Madrid, Spain)
- Gabor Fichtinger (PerkLab, Queen’s University , Kingston , Canada)
- Andras Lasso (PerkLab, Queen’s University , Kingston , Canada)
- Adam Rankin (Robarts Research Institute / Western University, Canada)
- Csaba Pinter (Ebatinca S.L., Las Palmas de Gran Canaria, Spain)
- Lucas Gandel (Kitware, France)
- Jean-Christophe Fillion-Robin (Kitware, USA)
- Simon Drouin (École de Technologie Supérieure , Montreal , Canada)
Project Description
Augmented Reality has increased its adoption in many areas with exciting benefits. Universidad Carlos III de Madrid (Madrid, Spain) has already worked in several medical projects based on AR (see their progress in https://biig-igt.uc3m.es/augmented-reality/). On these studies, they usually export information from Slicer to an alternative software (Unity).
The ultimate goal of this project is to develop a new 3D Slicer module that will allow to use augmented reality directly in 3D Slicer. With it, it will be possible to centralize the working process, at the time of benefiting from all Slicer tools.
Ebatinca S.L. in Las Palmas de Gran Canaria (Spain), has already started this project in collaboration with Kitware (France) and Universidad Carlos III de Madrid (Spain) and they all are currently developing OpenXR.
Objective
- Create OpenXR, a code based on the currently existing OpenVR so that the new mixed reality version is compatible with VTK.
Some links of interest:
- Writing a Holographic Remoting remote app using the OpenXR API
- Slicer Documentation on Augmented Reality and Virtual Reality support
Approach and Plan
- Develop a new Slicer extension that streams AR directly to HoloLens 2.
Progress and Next Steps
- OpenXR in VTK:
The WIP branch supporting Holographic remoting to stream VTK rendering inside the Hololens has been submitted here.
- Our github latest upload
- We confirmed the prototype developed by Lucas Gandel by streaming VTK to the HoloLens 2 via Holographic Remoting.
Illustrations
Here below you can find some AR implementations in health by Universidad Carlos III de Madrid (Madrid, Spain) in the past years:
HoloLens 2 in Orthopedic Oncological Surgeries:
Smartphone app to communicate with the patient and help him/her understand his/her condition:
Real-time guidance during Open Cranial Vault Remodeling using smartphone:
Needle Insertion Guidance for Sacral Nerve Stimulation using smartphone:
