Article · Applied Sciences · September 28, 2023
A Real-Time Dynamic Gesture Variability Recognition Method Based on Convolutional Neural Networks
Nurzada Amangeldy, Marek Miłosz, Saule Kudubayeva, Аkmaral Kassymova, Gulsim Kalakova, Lena Zhetkenbay
Abstract
Among the many problems in machine learning, the most critical ones involve improving the categorical response prediction rate based on extracted features. In spite of this, it is noted that most of the time from the entire cycle of multi-class machine modeling for sign language recognition tasks is spent on data preparation, including collection, filtering, analysis, and visualization of data. To find the optimal solution for the above-mentioned problem, this paper proposes a methodology for automatically collecting the spatiotemporal features of gestures by calculating the coordinates of the found area of the pose and hand, normalizing them, and constructing an optimal multilayer perceptron for multiclass classification. By extracting and analyzing spatiotemporal data, the proposed method makes it possible to identify not only static features, but also the spatial (for gestures that touch the face and head) and dynamic features of gestures, which leads to an increase in the accuracy of gesture recognition. This classification was also carried out according to the form of the gesture demonstration to optimally extract the characteristics of gestures (display ability of all connection points), which also led to an increase in the accuracy of gesture recognition for certain classes to the value of 0.96. This method was tested using the well-known Ankara University Turkish Sign Language Dataset and the Dataset for Argentinian Sign Language to validate the experiment, which proved effective with a recognition accuracy of 0.98.
More by Lena Zhetkenbay
- Intelligent Inclusive Navigation System for a University Digital EcosystemComputers · 2026
- Directional Asymmetry of Domain-Adversarial Adaptation in Cross-Corpus Speech Emotion RecognitionIEEE Access · 2026
- Development of an Emotion Recognition System to Monitor Children's Emotions in Preschool InstitutionsInternational Conference on Computer Science and Engineering · 2025
- Multi-objective Optimization Framework for Energy Efficiency and Production Scheduling in Smart Manufacturing Using Reinforcement Learning and Digital Twin Technology IntegrationInternational Journal of Industrial Engineering and Management · 2025 · Citations: 16
