Talk · IEEE International Conference on Smart Information Systems and Technologies · May 14, 2025

Development of a Translator for Kazakh Sign Language to Speech Using Gesture Recognition

Айгерим Айтим, Dariga Sattarkhuzhayeva, Aisulu Khairullayeva

Abstract

This paper presents the development of "DauysYm," an AI-powered translator that converts Kazakh Sign Language (KSL) into spoken language using dynamic gesture recognition. The project addresses communication barriers faced by people with hearing impairments in Kazakhstan. The main objective of this research is to develop an inclusive technological solution that enables real-time translation of KSL into speech to support the social integration of hearing-impaired individuals. The system integrates convolutional neural networks (CNNs), recurrent neural networks (RNNs), and attention mechanisms to analyze gesture sequences with high accuracy. Data was collected using OpenCV and MediaPipe, resulting in a structured dataset of dynamic gestures. A survey with 140 respondents confirmed the high societal need and user readiness for such a system. The results show promising training and validation performance, demonstrating the effectiveness of the proposed approach. "DauysYm" offers a scalable, inclusive solution that enhances communication and supports social integration, showcasing the potential of AI technologies in assistive applications tailored for the Kazakh context.

Hand Gesture Recognition SystemsHearing Impairment and CommunicationAssistive Technology in Communication and Mobilitysign language to speech translationgesture recognitionhearing loss

Development of a Translator for Kazakh Sign Language to Speech Using Gesture Recognition

Abstract

More by Aigerim Aitim