Talk · IEEE International Conference on Acoustics Speech and Signal Processing · April 21, 2026
Prototype Learning-Based Few-Shot Medical Image Segmentation Framework Using Clip-Aided U-Net Model
Saptarshi Pani, Bhumika Adhya, Vyacheslav V. Gulvanskii, Dmitrii Kaplun, Ram Sarkar
Abstract
Medical image segmentation is crucial for automated clinical diagnostics, but is often hindered by the scarcity of annotated data. To address this, we propose a novel few-shot learning-based segmentation framework that integrates the complementary strengths of U-Net, for precise boundary localization, and CLIP, for rich semantic generalization. Our method introduces a cross-attention fusion mechanism to align support prototypes, image embeddings, and task-specific textual prompts, thereby enabling robust segmentation from limited examples. Extensive experiments on PanNuke, GlaS and Kvasir-SEG datasets demonstrate that our model achieves state-of-the-art performance across 3-shot, 5-shot, and 10-shot settings. The source code of the proposed model and additional results are available at: https://github.com/SaptarshiPani/CLIP-based-U-Net/.
More by Dmitry Kaplun
- Codewithsayanjib/slm-systematic-review: v1.1Zenodo · 2026
- Microscopy Image Dataset of pulmonary vessels for Quantitative assessment of fibrosisScientific Data · 2026
- Microscopy Image Dataset of pulmonary vessels for Deep Learning-Based Quantitative assessment of fibrosisFigshare · 2026
- MA2MU-Net++: A Mahalanobis distance and Ackley function based multi-modal U-Net++ model for breast nuclei segmentation in histopathology imagesBiomedical Signal Processing and Control · 2026