NA-MIC Project Weeks

Back to Projects List

Helping LLMs view IDC images

Key Investigators

• Michael Halle (BWH, USA)

Project Description

The Imaging Data Commons is filled with imaging data, but idc-index and database searches only return textual data/metadata to LLMs. Modern LLMs have vision capabilities and can view selected images. I will complete a standalone non-interactive imaging utility that will give LLMs an image viewing capability for IDC data.

Objective

1. A standalone python CLI for displaying an image or mosaic from an IDC series.
2. A Claude Skill (or an extension of an existing skill) that teaches an LLM to look at IDC images.

Approach and Plan

1. Finish the CLI
2. Get feedback
3. Finish the Skill

Progress and Next Steps

1 Prototype “idc-series-preview” is written but needs knowledgable eyes on the project. It also needs some features like annotations added. This project could well be done using fancy DICOM viewer code, but for right now Python works fine for the prototype. 1 During Project Week, I added annotations to the python module and made it into a Claude Skill.

Python module/skill: https://github.com/mhalle/idc-series-preview

Skill download: https://github.com/mhalle/idc-series-preview/releases/latest/download/idc-series-preview.skill

Illustrations

“Find the bottom of the lungs in an IDC NLST series”:

Contrast Mosaic:

Background and References

No response