This breakout aims to identify the current state of VR capabilities in Slicer, identify areas of most need for improvement, and brainstorm about future potential applications of VR in Slicer.
VR is currently available in Slicer out-of-the-box via the SlicerVirtualReality extension. This extension uses the VTK OpenVR interface that provides input, tracking, and visualization capabilities. To use this, Steam and SteamVR must be installed and running (SteamVR installed from within Steam).
Preliminary work for AR support in Slicer is functional at SlicerAugmentedReality but needs improvements. Video passthrough infrastructure is available in VTK (last tested 8.2).
Tracking data is available for all OpenVR devices including headset, controllers, and generic trackers. Input from the controllers is available and the default interactor provides navigation, zoom, and selection capabilities.
The 3D view in Slicer can be rendered in compatible OpenVR headsets. Other views including 2D Slice, Chart, etc… view are not currently rendered to the headset.
The Leap Motion hand tracking device can be streamed into Slicer and visualized using the SlicerLeapMotion extension.
Tested OpenVR headsets include:
The SlicerVideoCameras extension provides a wrapper to OpenCV’s camera calibration techniques.
We have identified the following major categories as areas that need improvement in the current Slicer ecosystem.
It is still unclear which input methods will provide the easiest navigation and action capabilities in VR, but is most likely a combination of input methods.
Even once input methods have been decided, interaction guidelines for input are still a shifting target in VR. Using controllers, for example, does one use the point of the device as a selection point, or use the direction of the controller as a laser pointer? There are many such questions/decisions to be settled.