Edit this page

NA-MIC Project Weeks

Back to Projects List

Collaborative VR breakout session

Key Investigators

Project Description

Virtual Reality (VR) has a great potential to facilitate communication between clinicians. For example, VR can be used to collaboratively plan a surgical case by manipulating 3D models derived from preoperative scans. Slicer already includes most of the components required to build prototypes for this kind of applications:


  1. Review the current functionality of SlicerVR and PRISM Rendering
  2. Discuss planned near future developments in those modules
  3. Lay out a vision for the future of interactive VR in Slicer
  4. Establish a protocol for Collaborative VR
  5. Points for discussion:
    1. Future possibilities for interacting in VR: hand tracking, tool tracking
    2. Need to support different types of tracking in collaborative VR protocols
    3. Should SlicerVR have a fixed set of functionality and interaction paradigm or let users and/or developers choose.

Progress and Next Steps

  1. Potential application of SlicerVR: Learn dental anatomy (Sébastien Erckelbout)
  2. Short term changes:
    1. Implement spatial references (e.g. a Floor)
    2. Improve object selection
  3. Interaction
    1. Is it worth sharing a code base with Slicer Looking glass?
    2. For developers: need for more control over interaction: disable existing interaction and change behavior
    3. Long term: support for more complex VR controllers and hand tracking
    4. Have interaction settings for different scenarios (inside-out vs outside-in visualization)
  4. Collaboration
    1. Currently, collaboration is setup using OpenIGTLinkIF and carefully setting up scenes on both ends. Transforms have to be setup for controllers and HMD
    2. We need to facilitate (automate) connection and scene setup for collaboration
    3. It might be necessary to use a server technology to hold common state (Mike Halle suggested FireBase previously used)
    4. A matchmaking and communication solution to explore: Photon


Background and References



PRISMRendering doc