Deep Learning-Based Continuous Sign Language Recognition

Abstract

This study focuses on the development of a continuous sign language recognition system based on deep neural network models. A new Kazakh Sign Language (QazSL) dataset is created. DL models for continuous KazSL are developed, their accuracy and robustness under different environmental conditions are analyzed, and an optimized model algorithm to improve sign recognition processes are proposed. The main goal is to improve gesture recognition accuracy, account for gesture variability and environmental conditions, and promote the development of adaptive technologies for low-resource languages. This paper proposes a QazSL recognition system using an YOLOv8n and optimized 2DCNN models to improve accessibility for the hearing impaired. The optimized 2DCNN method includes optimal data preprocessing techniques and new training architecture, followed by model training and testing with precision, recall, and accuracy metrics. The proposed systems were trained using an opencourse K-RSL dataset with 5 signers and a newly created QazSL dataset, recorded by 7 signers. The test accuracy of gesture recognition are 98.12% for Yolov8n and 98, 57% for 2DCNN, indicating the robustness and capability of the models for realtime application. Certain issues, such as background variation and gesture consistency, were found to affect recognition under different conditions. This research contributes to the development of AI-based assistive technology to facilitate social inclusion and access to communication for deaf and hard-of-hearing people. By addressing the challenges identified in gesture recognition, this study paves the way for more reliable interactions between users and technology. Future work will focus on optimizing the model further to enhance its performance in varied environments and to expand its applicability across different languages and sign systems.

More by Azamat Serek

• COMPARATIVE ANALYSIS OF MACHINE LEARNING METHODS FOR DETECTING FRAUD IN TEXT MESSAGESNEWS OF THE NATIONAL ACADEMY OF SCIENCES OF THE REPUBLIC OF KAZAKHSTAN · 2026
• EXPLAINABLE MULTISPECTRAL ATTENTION NETWORK FOR SENTINEL-2 AGRICULTURAL LAND CLASSIFICATIONScientific Journal of Astana IT University · 2026
• SEMANTIC SEGMENTATION OF AGRICULTURAL FIELD ANOMALY PATTERNS IN AERIAL IMAGES USING U-NETHerald of Kazakh-British technical university · 2026
• From Detection to Maps: A Review of Automated Urban Tree Mapping Using UAV and High-Resolution Satellite DataJournal of Imaging · 2026