Advancing Orthopaedic Imaging: Self-Supervised Vision Transformers for 3D Reconstruction of Knee Magnetic Resonance Imaging

Abstract

The main aim of this research is to develop and evaluate a self -supervised learning model based on Vision Transformers for three-dimensional reconstruction of knee Magnetic Resonance Imaging. Three-dimensional reconstruction of hard tissues is an essential procedure for visualization and structural analysis of bones, both in clinical and research settings. Traditional compressed sensing approaches rely on handcrafted priors and iterative solvers, while supervised convolutional neural networks require large datasets of fully sampled Magnetic Resonance Images, which are costly to obtain. To address these limitations, this paper proposes the self-supervised framework based on Vision Transformers architecture along with the integrated convolutional neural network decoder for reconstructing volumetric information from an under-sampled Magnetic Resonance Imaging data. We propose a self-supervised learning framework using Vision Transformers for three – dimensional knee magnetic resonance image reconstruction. Our method leverages masked patch pre-training on a custom dataset of proton density fat-saturated knee magnetic resonance imaging scans, followed by volumetric decoding with a lightweight three- dimensional convolutional neural network. Results demonstrate competitive reconstruction quality with Dice score of 0.87, IoU of 0.83, and recall of 0.93, showing that self-supervised vision transformers effectively capture long-range dependencies and volumetric continuity. This study highlights the potential of Vision transformer-based self ‒ supervised learning frameworks to overcome the limitations of traditional supervised deep learning approaches. The proposed method demonstrated that it can reduce dependence on large fully sampled datasets and support accurate three -dimensional reconstructions for clinical and research applications.

More by Damira Mussina

• Multi-Modal Data Fusion Using Deep Neural Network for Condition Monitoring of High Voltage InsulatorIEEE Access · 2020 · Citations: 61
• IN-YOLO: Real-Time Detection of Outdoor High Voltage Insulators Using UAV ImagingIEEE Transactions on Power Delivery · 2019 · Citations: 312
• High voltage outdoor insulator surface condition evaluation using aerial insulator imagesHigh Voltage · 2019 · Citations: 51
• Deep-Learning-Based Approach for Outdoor Electrical Insulator InspectionModeling and optimization in science and technologies · 2019 · Citations: 2