Talk · Conference on Empirical Methods in Natural Language Processing · January 1, 2026

Do LLMs Cross Alphabets? A Cross-Script Audit of LLMs Across Four Kazakh Latin Proposals

Galammadin Askar, Arailym Mukhametkali, Akylbek Maxutov, Zhanat Makhataeva, Nūrali Medeu, Hüseyin Atakan Varol

Abstract

Kazakhstan is transitioning the Kazakh language from Cyrillic to a Latin alphabet, with four official Latin proposals issued since 2017.How well do current large language models, trained almost entirely on Cyrillic Kazakh, handle this script change?We evaluate 13 models spanning general-purpose open-weight, Kazakh-adapted, and proprietary systems on three Kazakh benchmarks (KazMMLU, Kaz-Culture, GSM8k-Kazakh) rewritten in all four Latin variants.The mean accuracy drop from Cyrillic to Latin is 16.8 points, ranging from 1.4 (GPT-5) to 30.4 (Sherkala-8B).Contrary to the natural tokenization-explosion hypothesis, fertility analysis shows that Latin Kazakh requires no more tokens per word than Cyrillic for most tokenizers.Subword fragmentation therefore contributes to the drop but does not primarily explain it.Kazakh-adapted models drop hardest, suggesting that Cyrillic-only adaptation entrenches script-specific routes rather than building script-agnostic competence.Promptlevel mitigations (hint, mapping) recover only 1 point on average.On the other hand, LoRA fine-tuning recovers up to +20 points, but only for sufficiently large reasoning-capable bases.Generative AI will play a key role in the script transition.Models serving Kazakh users must reliably process Latin script, which our findings show cannot be achieved at inference time.Addressing this gap requires incorporating Latin Kazakh data during pretraining or large-scale continued pretraining, rather than relying solely on prompt or adapter-level solutions.

Natural Language Processing TechniquesTopic ModelingEthics and Social Impacts of AIlarge language modelsorthographic variationKazakh language

Do LLMs Cross Alphabets? A Cross-Script Audit of LLMs Across Four Kazakh Latin Proposals

Abstract

Related publications