Bridging dialectal variations in Uzbek texts: A comparative evaluation of modern approaches

Abstract

In this paper, we propose a solution to the problem of identifying dialect words in Uzbek texts using neural network methods focused on the contextual representation of lexical units. As the main comparison tools, the authors chose models based on the spaCy library, as well as an architecture combining bidirectional LSTM and a convolutional neural network (CNN). The authors note that existing solutions are not able to either generalize the original data or analyze the context of sentences for the most accurate standardization of dialect words into formal equivalents. As a result of training the models, it was found that the spaCy-based model achieved such indicators as accuracy of 90%, recall of 89%, and f1-score of 90%. While the biLSTM+CNN bundle achieved such values as accuracy of 92%, recall of 91%, and f1-score of 92%. Moreover, the authors cited existing solutions, talked about their approaches and the reasons why they cannot cope with the task under study.

More by Nilufar Abdurakhmonova

• A model of a universal linguistic knowledge graph for Turkic languagesArctic XXI century · 2026
• Machine Learning-Based Topic Classification of Economic and Financial Texts in UzbekInternational Conference on Computer Science and Engineering · 2026
• Building a Phraseological Ontology of Uzbek Idioms and Evaluating Similarity Measures for Their Detection in TextInternational Conference on Computer Science and Engineering · 2026
• Developing Morphological and Syntactic Analysis Models for Karakalpak as a Low-Resource Turkic LanguageInternational Conference on Computer Science and Engineering · 2026