Back to Projects List
Cinematic rendering in Slicer leveraging VTK Physically Based Rendering (PBR)
- Shreeraj Jadhav, Kitware, USA
- Jiayi Xu, Kitware, USA
- Jean-Christophe Fillion-Robin Kitware, USA
- Andras Lasso (PerkLab, Queen’s University, Canada)
- The goal of this project is to identify and evaluate Kitware’s previous efforts on cinematic rendering and how these can be leveraged in Slicer.
Slicer internally uses VTK as its rendering engine to display the 2D and 3D viewports in its interface. Since release 9.0, VTK has more enhanced rendering capabilities such as the Physically Based Rendering (PBR) lighting model and the integration of ray tracing backends such as the Nvidia OptiX and intel’s OSPRay engine. These capabilities are already being leveraged in Paraview as discussed in the following blog posts here and here.
- Primary focus will be on VTK’s PBR capabilities and ray tracing backends (OptiX, OSPRay). Since Slicer already uses VTK rendering pipeline, and it will be significantly easier to incorporate/leverage these capabilities from within Slicer.
Approach and Plan
- Review accessible offerings on cinematic rendering (VTK PBR, VTK backends: OSPRay and OptiX, omniverse, etc) that can be utilized inside Slicer.
- Develop prototypes showing how these can be enabled and used in Slicer.
- Start with surface rendering (PBR, ambient occlusion, global ilumination)
- Investigate what capabilities can be enabled for volume rendering (perhaps OSPRay).
- Tradeoffs: Performance vs Image Quality without degrading user experience.
- Changes to user interface, parameters tuning, simplification.
- Review existing Slicer modules (such as Light Module) that enhance Slicer’s rendering capabilities and evaluate how these can be included in the current effort.
- Evaluate how integrating these in Slicer will affect other modules such as LookingGlass, OpenXR, etc.
Progress and Next Steps
vtkSSAOPass class to generate ambient occlusion (AO) for volumes:
- Initial attempt encountered many OpenGL State errors
- Volume mapper cannot directly work when AO pass is enabled, need further investigations to understand how this could be done.
vtkSSAOPass to create local ambient occlusion (LAO) implementation for volumes:
ComputeKernel() in vtkSSAOPass can be modified to adapted for LAO of volumes [Jiayi]
- Possibly, only compute an occlusion volume once (pre-compute), use this for shading in raycaster
Through discussions with the VTK team (Timothee Chabat, Mathieu Westphal), we identified another feature which could help improve shading in volume rendering:
- an accurate ambient occlusion effect can be achieved with the current VTK master, setting the
0 and setting
VolumetricScatteringBlending to something
Image Courtesy: Gaspard Thevenon
Background and References
- Models: In recent Slicer versions, PBR interpolation can be selected and PBR material properties can be edited.
- Lights (in Sandbox extension): it can configure lighting, PBR, image based lighting, ambient shading (SSAO)
Slicer Discourse References:
- VTK PBR https://www.kitware.com/vtk-pbr/
- PBR integration in Paraview https://www.kitware.com/physically-based-rendering-improvements-in-paraview/
- Related merge request for VTK https://gitlab.kitware.com/vtk/vtk/-/merge_requests/5584