NA-MIC Project Weeks

Back to Projects List

SlicerUltrasound Extension Development - New features

Key Investigators

• Maha Kesibi (Queen's University, Canada)
• Tina Kapur (BWH, USA)
• Tamas Ungi (Queen's, Canada)
• Fahimeh Fooladgar (UBC, Canada)
• Shreyas Puducheri (BWH, USA)
• Matt Alves (BWH, USA)
• Caroline Schissel (Lahey, USA)
• David Dinh (SlicerUltrasound Team, USA)
• Atin Malaviya (SlicerUltrasound Team, USA)
• Sam Horvath (Kitware, USA)
• Jean-Christophe Fillion-Robin (Kitware, USA)

Project Description

AnnotateUltrasound is a 3D Slicer extension that enables structured sector annotation of lung ultrasound video clips, focusing on features such as pleura lines and B-lines. It provides an intuitive interface for frame-by-frame annotation, supports multiple raters, and saves annotation data for future research and machine learning.

We collaborate closely with several physicians who use AnnotateUltrasound in their clinical and research workflows. Their feedback directly shapes the module’s features and usability, ensuring a user-centered design. We regularly incorporate their suggestions for new features and improvements based on their real-world experiences.

This week, we focused on incorporating user-requested features to improve efficiency, usability, and comparative analysis of annotations.

More about the module: https://github.com/SlicerUltrasound/SlicerUltrasound

Objective

1. Frame-by-Frame Pleura Percentage Comparison
   • Display pleura percentage per frame.
   • Highlight the frame with the highest pleura coverage for each rater in the rater table.
2. Annotation Time Tracking
   • Track and display time each rater spends annotating, to support workload analysis and training evaluation.
3. Adjudication Mode
   • Introduce an adjudicator workflow for validating or invalidating rater annotations and generating final consensus annotations.
4. Label Annotation Comparison
   • Add label data to the rater table to allow comparison of both line and label annotations.
5. Improved Line Endpoint Visualization
   • Refine endpoint markers to support more precise annotations.

Approach and Plan

• Update the AnnotateUltrasound logic to compute and store pleura percentages per frame and clip.
• Extend UI tables to display these values, with live updates as annotations change.
• Integrate a timer using QTimer and idle detection (from the UserStatistics module) to track annotation time and store it in JSON.
• Modify endpoint visuals for clearer frame annotation.
• Add an adjudicator mode for validating annotations across raters, including new UI tools and changes to the annotation file structure to store validated results

Progress and Next Steps

Completed

• Annotation Timer
  • Tracks time per rater, auto-pauses on inactivity
  • Displays timer in MM:SS format
  • Saves to annotation JSON
• Pleura Percentage Display
  • Shows per-frame pleura percentages in annotation table
  • Highlights max pleura frame in rater table with frame index
  • Values stored in annotation JSON
• Added Adjudicator Mode for validating annotations across raters
  • UI tools for validate/invalidate actions (+ keyboard shortcuts)
  • Saves to .adjudication.json file with adjudicator metadata (status, timestamp)
  • Updates schema to support final validated annotations

In Progress

• Refine UI for adjudication visibility and toggles
• Add the Sandbox/UserStatistics module as a dependency
• Testing edge cases and fine-tuning timer and pleura calculations
• Adding the label annotation to the UI in the rater table
• Get user feedback from clinicians for final usability tweaks

Illustrations

Pleura percentage saving per frame and populating rater table:

Timer demo:

Label annotation and populating rater tbale:

Adjudication video:

https://github.com/user-attachments/assets/d3cb237e-92d7-4a67-87ff-18c41fe003cd

Current keyboard shortcuts:

Background and References

• Source Code: https://github.com/SlicerUltrasound/SlicerUltrasound
• Related Extension: Sandbox – UserStatistics module

No response