NA-MIC Project Weeks

Back to Projects List

Vox2SegLAM: Protocol-Guided Subcortical Segmentation in 3D Slicer

Key Investigators

• Ahmed Rekik (École de technologie supérieure, Montréal, Canada)
• Jarrett Rushmore (Center for Morphometric Analysis, Massachusetts General Hospital, Boston, USA)
• Sylvain Bouix (École de technologie supérieure, Montréal, Canada)

Project Description

3D SlicerVox2SegLAM is an extension that brings an AI-assisted, protocol-guided workflow to subcortical brain segmentation and landmarking. A joint CNN+GNN model predicts 26 subcortical structures and 20 anatomical landmarks directly from a T1 MRI, following the Harvard-Oxford Atlas (HOA 2.0) neuroanatomical protocol.

Objective

1. Develop an open-source 3D Slicer extension that automatically predicts subcortical segmentation (26 structures) and 20 neuroanatomical landmarks from a single T1 MRI, following the Harvard-Oxford Atlas (HOA 2.0) protocol.
2. Give the expert fast, intuitive tools to correct AI predictions directly in Slicer — drag landmarks, add/remove landmarks, edit segments — without leaving the scene.
3. Enforce anatomical consistency automatically through an explicit, human-readable rule engine that re-applies HOA2.0 protocol constraints after every manual edit, minimizing the time needed to produce gold-standard reference annotations.

Approach and Plan

1. Architecture: a CNN extracts multi-scale features, shared with a cascaded GNN that refines a template landmark graph onto the patient’s anatomy in 4 steps.
2. Segmentation: landmarks drive a rule-based post-processing step that splits coarse CNN groupings into the 26 target structures under anatomical constraints.
3. Slicer module: package model + rule engine into Vox2SegLAM — inference, interactive editing of landmarks/segmentation, rule re-application, export.
4. Validate on held-out and out-of-domain scans.

Progress and Next Steps

Done:

1. Trained the joint CNN+GNN model: mean Dice = 0.908 ± 0.010 across 26 subcortical structures .
2. Achieved a mean landmark localization error of 1.4 mm under strict point-to-point evaluation, improving to 1.1 mm under an anatomically-tolerant evaluation that accounts for legitimate placement ambiguity (e.g., any voxel along the correct structure boundary or within the correct coronal region counts as correct), across the 20 predefined HOA2.0 landmarks.

Next steps (at/after PW45):

1. Built the core of the Slicer extension: inference adapter, protocol rule parser for the HOA protocol and the geometry engine (plane and landmarks fitting.
2. Validated qualitatively on in-domain test data and on out-of-domain scans, confirming anatomically plausible predictions outside the training distribution.
3. Work with Dr. Rushmore to refine the protocol rule set and run a usability/annotation-time study comparing manual vs. AI-assisted annotation.

Illustrations

• Figure 1. Overall architecture: shared CNN encoder–decoder with multi-scale feature exchange feeding a 4-step cascaded GNN that refines landmark positions from a template graph.

• Figure 2. Per-region Dice coefficients (26 structures), mean Dice = 0.908.

• Figure 3. Per-landmark mean localization error (mm): strict (“Classic”, 1.4 mm) vs. anatomically-tolerant evaluation (1.1 mm).

• Figure 4. Visual example on in-domain test data:
• Figure 5. Visual example on out-of-domain data:

Results

During this Project Week, we achieved the following:

• Remote hub deployment: The Vox2SegLAM pipeline was successfully deployed into a remote hub-based infrastructure, enabling users to run inference online without any local installation. The system is functional, and optimization of processing time is ongoing. Special thanks to Martin Bellehumeur for his collaboration on this part.

• Expert feedback and model evaluation: We had the opportunity to receive feedback from Professor Jarrett Rushmore on the automatic predictions generated by the pipeline. This allowed us to identify the current limitations of the model, which will be extremely valuable for future refinements.

• 3D Slicer extension: We developed a 3D Slicer extension that allows users to run the Vox2SegLAM pipeline directly and obtain both subcortical segmentations and anatomical landmark predictions. Experts can then interactively refine the results to produce segmentations that respect neuroanatomical protocols and anatomical constraints. The framework is designed to be flexible and support different protocols, with the current implementation configured by default for the Harvard-Oxford Atlas protocol.

• Video. Full workflow demo — hub-based remote inference deployment (VolView/CastInterface) followed by the Vox2SegLAM pipeline end-to-end:

</video>

Background and References

1. Rekik et al., “Automatic Landmark-Based Segmentation of Human Subcortical Structures in MRI,” arXiv:2605.14221, 2026. Link
2. Bongratz et al., “Vox2Cortex: Explicit Cortical Surface Reconstruction with Geometric Deep Learning,” arXiv:2203.09446, 2022. Link
3. Rushmore et al., “Anatomically curated segmentation of human subcortical structures in high-resolution MRI,” Frontiers in Neuroanatomy, 16:894606, 2022. Link
4. Rushmore & Harvard-Oxford Atlas Group, HOA Subcortical Brain Structure Segmentation Manual, Harvard University, 2021. Link