Back to Projects List
VTK.js Multi-volume Volume Rendering
The Open Health Imaging Foundation (OHIF) Viewer uses VTK.js for multi-planar reformatting (MPR). We use thin-slice volume rendering to implement MPR in the OHIF Viewer. In order to support PET/CT image fusion with overlaid segmentation label maps, we need to implement multi-volume rendering in VTK.js. At present, multi-volume rendering sort of works but only for two volumes.
Our goal is to add proper support for multi-volume volume rendering to VTK.js.
Approach and Plan
- Evaluate how VTK C++ implements multi-volume rendering
- Clean up terminology in VTK.js’ RenderWindow
Progress and Next Steps
- Held a preparation tcon with the Kitware team to review status and goals.
- We built a multi-volume mapper class which uses a dynamically created shader based on the number of volumes, and alpha blends across volumes along the ray.
- Next steps:
- Fix clipping range computation from bounds around actors and test with real data
- Fix alpha blending, opacity is not correct
- Fix gradient opacity shading
- Test / fix multi-component rendering per volume
- Document all the variables and the approach
- Stack opacity and color textures across volumes. This would allow us to get to 12 volumes instead of 4.
Plans from early meetings
Notes here: https://docs.google.com/document/d/1160z3fKJB6JfmT_EAlPRbqA5dD9B5iNfe08JctMUWOE/edit?usp=sharing
- Current VolumeMapper will remain in place, we will add a separate optional path that enables the MultiVolumeMapper class
- Targeting WebGL2 Only
- Colour and Opacity lookup tables for each volume will be stacked into 2D textures (num volumes * width of largest LUT)
- Since we include a jitter texture, this means we could have up to 13 volumes, since Chrome WebGL has a maximum of 16 textures.
- Blend Modes other than Composite (e.g. MIP) will not be available in the multi-volume VolumeMapper
- Labelmap outline rendering will not be available in the multi-volume VolumeMapper
- Sampling distance will be computed from the minimum sample distance across the provided volumes.
- Vertex shader will need to be provided a bounding box of the union of all volumes in space. For now it can use the entire canvas.
- Work in progress branch is here: https://github.com/Kitware/vtk-js/compare/master…swederik:multivolume
This is what rendering two cubes currently looks like in VTK.js. It should turn purple at the intersection of the cubes if they were both being sampled and mixed along each step of the ray.
Work-in progress Multi-volume Renderer
Background and References