Talk · IEEE International Symposium on Biomedical Imaging · April 8, 2026

Segment Anything Model With Refined-Prompts for Multimodal Cross-Data Few-Shot Medical Image Segmentation

Bhumika Adhya, Saptarshi Pani, Irina I. Shpakovskaya, Dmitrii Kaplun, Ram Sarkar

Abstract

Medical image segmentation under limited supervision remains a challenging problem, particularly in cross-data scenarios, where shifts in data distribution hinder model generalization. In this work, we propose a multimodal cross-data few-shot segmentation framework that synergistically integrates the visual priors of the Segment Anything Model (SAM) with the semantic reasoning capabilities of Contrastive Language-Image Pre-Training (CLIP) through a cross-attention fusion module, while a U-Net-based Adaptive Mask Refiner aided with a Post-Processing module enhance boundary precision and anatomical consistency. Extensive experiments across different benchmark datasets, including Kvasir-SEG, KvasirCapsule-SEG, ISIC 2016, and PH2, demonstrate consistent improvements in Dice Score Coefficient (DSC) and Intersection over Union (IoU), validating that the framework adapts effectively to domain-specific medical data without requiring extensive annotations. The source code of the proposed model and additional results are available at: https://github.com/SaptarshiPani/SAM-CLIP/

Medical Image Segmentation TechniquesDomain Adaptation and Few-Shot LearningAdvanced Neural Network ApplicationsSegment Anything Modelmultimodal fusionCLIP

Segment Anything Model With Refined-Prompts for Multimodal Cross-Data Few-Shot Medical Image Segmentation

Abstract

More by Dmitry Kaplun