Edit this page

NA-MIC Project Weeks

Back to Projects List

Development of Deep Learning Segmentation for Spines with Metastatic Disease

Key Investigators

Project Description

We have labeled CT data sets for lumbar and/or thoracic as well as full spine columns for patients at baseline. For a good # of patients, we also have 3 and 6m follow-up CT, but these are not yet labeled. It would be great to get some help/advice regarding how to speed up the segmentation for the labeling and extraction of volume information from the masks. The segmented volumes are needed for the analytical and computational modeling pipeline as part of a collaboration with MIT. Lytic spines present a special segmentation challenge because the interior of vertibrae have degraded and appear like surrounding tissue in CT scans.


  1. Objective A. Setup MONAI-based neural network training on the labeled scans.
  2. Objective B. Investigate using MONAILabel as a possibly better way to annotate new scans.
  3. Objective C. Train a DNN on the labeled data. Can this model be used in MONAILabel?
  4. Objective D. Find a path forward to better annotation using DL-assisted segmentation.

Approach and Plan

  1. Start with publically available VerSe spine dataset.
  2. Evaluate MONAILabel from Andres using model trained on VerSe as a segmentation tool for these lytic spines. How good is segmentation?
  3. Consider fine-tuning model in MONAILabel with lytic spines from our dataset
  4. Train model on only our lytic spines for comparison
  5. Install MONAILabel on GPU hardware at BI to setup for better annotation after project week

Progress and Next Steps

  1. Received permission to share a few cases with collaborators
  2. Acquired GPU hardware. Evaluated installing Linux, WSL, and Conda-on-windows. Settled on Windows
  3. Andres Diaz-Pinto used the VerSe spine dataset to create a MONAILabel vertebrae segmentation app
  4. Curt was able to install and run MONAILabel locally
  5. Andras Lasso helped with a script to fill internal holes in the vertebrae
  6. Segmentation Editor performance is improved with the VTK9 version of Slicer and Andras engineering fixes
  7. Curt started preparing for ML training of a network using Ron’s improved annotations, but this is still in progress.
  8. Next Steps: Improve segmentations for the remaining annotated spines and use in MONAILabel.
  9. Next Steps: identify any other collaborators interested to continue working on this project; eventually expanding to include nearby muscle and disks

Spine segmentation protocol for training data generation

This protocol can be used to quickly and accurately segmentat the spine (each vertebra in a separate segment; no internal holes), which can be used as training data for deep learning based segmentation:


Here is an example of how the degraded vertebrae look in CT scans and the challenges of segmentation in Slicer. Note the interior holes because of the heterogeneous nature of the lytic bone.

Sample lytic vertebrae

Because the bone in the spines is degraded from cancer, it is difficult to get a good segmentation. Here is the result of several hours of manual segmentation:

Segmentation in Slicer

During the project week, we received consulting assistance on segmentation techniques from Andras Lasso and Rudolf Bumm. After Andras’ wrote us a post-processing script, the final result of registration is much better. These annotations can be used for neural net training:

improved spine annotation

Here is the MONAILabel server and Slicer Module running on Curt’s workstation. The app created annotations of vertebrae automatically created by a neural network trained using the VerSe spine dataset. This app works for interactively improving the segmentation through editing (using the deepedit algorithm):

MONAILabel vertebrae app

Development was started for training a standalone neural network model, also using the MONAI framework. Here is a sample training image showing a portion of the spine vertibrae and the corresponding label going into the neural network for training. This shows why it was critical to achieve solid annotations for the interior portions of the vertebrae:

Monai spine training image

Background and References

VerSe: A Vertebra Labelling and Segmentation Benchmark.