A Systematic Benchmark of Machine Transliteration Models for the Tajik-Farsi Language Pair: A Comparative Study from Rule-Based to Transformer Architectures

Abstract

This paper presents a systematic comparative benchmark of machine learning architectures for transliteration between the Tajik (Cyrillic script) and Persian (Arabic script) languages. A central contribution is the creation and open release of a unique, multi-domain parallel corpus comprising 328,253 sentence pairs aggregated from heterogeneous sources, including classical poetry, diplomatic texts, lexicographic data, and crowdsourced projects. To enable fully reproducible experiments under limited computational resources, a representative stratified subset of 40,000 pairs has been formed and is made publicly available together with the complete dataset. The study evaluates six model classes under a unified protocol: a rule-based baseline, an LSTM with attention, a character-level Transformer, a G2P Transformer trained from scratch, large pre-trained multilingual models (mBART, mT5 with LoRA), and the byte-oriented ByT5 model. The experimental results demonstrate that ByT5 achieves state-of-the-art performance with chrF++ scores of 87.4 for Tajik→Farsi and 80.1 for the reverse direction. Notably, the proposed G2P Transformer, trained exclusively on the 40,000-pair subset, significantly outperforms the substantially larger mBART model in the forward direction (72.3 vs. 62.2 chrF++), while subword-based models (mT5) fail almost completely, underscoring the necessity of character- or byte-level representations for this task. All collected data, source code, and model outputs are released under open licenses, thereby establishing a FAIR (Findable, Accessible, Interoperable, Reusable) benchmark for cross-script NLP in low-resource Central Asian languages.

More by Mullosharaf Arabov

• Parameter-Efficient Adaptation of Large Language Models to a Low-Resource Agglutinative Language: A Corpus, Datasets, and a Reproducible Benchmark for the Bashkir LanguagearXiv · 2026
• A comprehensive study of Tatar text classification methods: from corpus creation and Tatar2Vec embeddings to benchmark establishmentModeling and Analysis of Information Systems · 2026 · Citations: 1
• A Comprehensive Analysis of Arabic Natural Language Processing Research: Trends, Topic Evolution, and Research Gaps -- A Bibliometric and Topic-Based StudyarXiv · 2026
• Large Language Models for Low-Resource Languages: A Conceptual Framework for an Electronic Explanatory Dictionary of the Tajik LanguagearXiv · 2026