Edit this page

NA-MIC Project Weeks

Back to Projects List

ROS-MED: Integration of 3D Slicer and ROS2 for Image-Guided Robot-Assisted Interventions

Key Investigators

Project Description

The ultimate goal of this project is to provide a software platform to integrate medical image computing software (3D Slicer) into a system for image-guided robot-assisted interventions, in which 2D/3D medical images are used for planning, navigation, monitoring, and validation. Examples of such robot-assisted systems include image-guided robotic needle-guide systems and surgical CAD/CAM systems. Those systems often require a wide range of image computing capabilities such as segmentation of anatomical structures, registration of multiple images, 2D/3D image visualization, image-based planning, and data sharing with the robot controller and the hospital’s picture archiving and communication systems (PACS). Integration of a solid medical image computing platform into a robotic system is becoming more important than ever with the growing interest in AI-based treatment planning and guidance.

However, the engineering effort to implement those features is often underestimated in academic research due to limited engineering resources or the scope of the project. Fortunately, many of those features have already been implemented and validated in the research community and often distributed as open-source software. Therefore it has become essential for academic researchers to take advantage of those existing tools and incorporate them into their own research instead of reinventing the wheel.

Our team has been integrating 3D Slicer and Robot Operating System using OpenIGTLink to achieve the above goal following our first project, and recently received a grant from National Institutes of Health (2R01EB020667). In this year, we will focus on transition to the new ROS platform (ROS2).

Objective

  1. Objective A. Explore ROS2 as a potential platform for our study on image-guided model-driven needle placement robot.

Approach and Plan

  1. Run ROS2 on a universal robot arm (UR-10) at JHU.
  2. Prototype a new version of ROS-IGTL-Bridge
  3. Display a 3D model of the UR-10 on 3D Slicer, and synchronize its posture with the robot by sending the transform of each link.

Progress and Next Steps

First meeting (1:00pm on Discord)

Kick-off meeting (2:30pm on Discord)

ROS 2 testing on other environment

Tamas tried installing ROS2 on windows, but was not straightfoward. It might not be a viable solution for the workshop tutorial.

The original interface for ROS1 (ROS_IGTL_Bridge) has been modified for ROS2. The modified interface is now working with ROS 2 with limited capability (only supports text, transform, and point)

Setup ROS 2 and 3D Slicer on universal robot (UR-10e)

Approach:

Slicer Scene

Results:

Background and References

Acknowledgement

This work is supported by NIH R01EB020667 (MPI: Tokuda, Krieger, Fuge, Leonard).