Skip to content
DSML.KZ
Publications

Article · IEEE Transactions on Geoscience and Remote Sensing · January 1, 2025

Byte Latent Mamba With State Space and Knowledge Distillation for Hyperspectral Image Classification

Muhammad Imran Ahmad, Manuel Mazzara, Salvatore Distefano, Adil Khan

Manuel Mazzara
Open via DOI

Abstract

Hyperspectral image classification (HSIC) is a challenging task due to the high dimensionality of hyperspectral data, the complex interplay of spatial and spectral features, and the scarcity of annotated samples. Existing approaches, mainly based on tokenization-based feature extraction, introduce artificial segmentation, increasing computational cost, and may lead to information loss. To address these issues, a novel framework, Byte Latent Mamba with Knowledge Distillation (BLM-KD), overcoming explicit tokenization by directly learning byte-level spectral-spatial representations from raw hyperspectral data, is proposed. The Byte Latent Mamba architecture learns compact and expressive byte-level features through an end-to-end convolutional encoder, preserving spectral continuity and spatial structure. A structured State Space Model (SSM) is integrated to model long-range spatial-spectral dependencies efficiently via learned dynamic state transitions. Additionally, an adaptive knowledge distillation (KD) strategy is adopted, where a high-capacity teacher model selectively transfers salient features to a lightweight student model, driven by a temperature-controlled weighting schedule. This ensures robust generalization with reduced model complexity. A patch-based preprocessing scheme also excludes irrelevant zero-labeled samples, refining the training process. Extensive experiments conducted on multiple real-world hyperspectral benchmarks demonstrate that BLM-KD outperforms existing state-of-the-art methods in both classification accuracy and computational efficiency.

Remote-Sensing Image ClassificationAdvanced Neural Network ApplicationsSpectroscopy and Chemometric Analyseshyperspectral image classificationstate-space modelknowledge distillation

More by Manuel Mazzara