Skip to content
DSML.KZ
Publications

Talk · Mediterranean Conference on Embedded Computing · June 10, 2025

VEGAN: A Vision-language and Edge-enhanced GAN-based Microscopic Medical Image Segmentation Model

Gouranga Maity, Souvik Pramanik, Diptarka Mandal, Dmitrii Kaplun, Vyacheslav V. Gulvanskii, Ram Sarkar

Dmitry Kaplun
Open via DOI

Abstract

In this paper, we introduce VEGAN: a vision-language and edge-enhanced GAN-based Microscopic Medical Image segmentation model, a novel framework that combines Attention U-Net with a Pix2Pix Generative Adversarial Network (GAN), incorporating a vision-language model (VLM), called CLIP, within skip connections of the U-Net model. In our approach, the Attention U-Net takes both the input image and an edge map to enhance spatial feature extraction. Additionally, the encoder part of the generator integrates Convolutional Block Attention Modules (CBAM) to refine feature representations by emphasizing informative spatial and channel-wise features. The Pix2Pix GAN framework further refines segmentation results, with the generator utilizing the modified Attention U-Net and the discriminator employing PatchGAN to improve structural accuracy. The inclusion of CLIP-based skip connections enhances semantic understanding, aiding in feature retention, and better generalization. The model is evaluated on a publicly available dataset, called MoNuSeg and CNS for nuclei segmentation. Additionally, to ensure the robustness, it has been evaluated on the PH2 dataset for skin lesion segmentation. Experimental results demonstrate that our model achieves a Dice scores of 79.24%, 99.56% and 92.54% on MoNuSeg, CNS and PH2 datasets, respectively, which are better than many recently proposed models found in the literature. The improved results underscore the effectiveness of VEGAN architecture in handling the complexity of microscopic medical images, making it a valuable tool for advancing automated cellular analysis in biomedical research. Code and additional results will be found at GitHub .

Medical Image Segmentation TechniquesCell Image Analysis TechniquesAI in cancer detectionvision-language modelsCLIPAttention U-Net

More by Dmitry Kaplun