Preprint · arXiv · November 25, 2025

KyrgyzBERT: A Compact, Efficient Language Model for Kyrgyz NLP

Adilet Metinov, Kudakeeva, Gulida M., Kabaeva, Gulnara D.

Read on arXiv PDFDOI 10.48550/arxiv.2511.20182

Abstract

Kyrgyz remains a low-resource language with limited foundational NLP tools. To address this gap, we introduce KyrgyzBERT, the first publicly available monolingual BERT-based language model for Kyrgyz. The model has 35.9M parameters and uses a custom tokenizer designed for the language's morphological structure. To evaluate performance, we create kyrgyz-sst2, a sentiment analysis benchmark built by translating the Stanford Sentiment Treebank and manually annotating the full test set. KyrgyzBERT fine-tuned on this dataset achieves an F1-score of 0.8280, competitive with a fine-tuned mBERT model five times larger. All models, data, and code are released to support future research in Kyrgyz NLP.

Topic ModelingNatural Language Processing TechniquesSentiment Analysis and Opinion MiningKyrgyz languagelow-resource languagesmonolingual language models

KyrgyzBERT: A Compact, Efficient Language Model for Kyrgyz NLP

Abstract

More by Adilet Baktiiarovich Metinov