Back to Projects List
Up to date, there has been a lack of software infrastructure to connect 3D Slicer to any augmented reality (AR) device. This project presents a novel connection approach using Microsoft HoloLens 2 and OpenIGTLink. This project has been developed in collaboration of Universidad Carlos III de Madrid (Madrid, Spain) and Perk Lab in Queen’s University. The current solution is implemented in a 3 elements system. It is composed by A Microsoft HoloLens 2 headset, the Unity software, and the 3D Slicer platform.
Create a universal module in 3D Slicer that sends all types of messages via OpenIGTLink.
We have already developed an application that transfers geometrical transform and image messages between the platforms. It displays CT reslices of a patient in the AR device. The user wearing the glasses can manipulate the CT plane to see different perspectives. The application was build for pedicle screw placement planning.
The final version will be able to transfer any type of messages. To do so, we have to create necessary scripts in 3D Slicer and also in Unity (C#).
Check out our app in this GitHub repository. This repository contains all the resources and code needed to replicate our work in your computer.
Transfer of geometrical transforms from HoloLens 2 to 3D Slicer:
Transfer of images from 3D Slicer to HoloLens 2:
Use this system for multiple HL2 interaction: