Skip to content
DSML.KZ
Publications

Talk · AIP conference proceedings · January 1, 2025

A comparative study of annotation schemes for named entity recognition in Uzbek legal texts

Davlatyor B. Mengliev, Nilufar Z. Abdurakhmonova, Raima Kh. Shirinova, Bahodir B. Ibragimov, Khudayberganova Dildora Kulmamatovna, Mohinur Ilhom qizi Suyunova

Nilufar Abdurakhmonova
Open via DOI

Abstract

Developing effective named entity extraction systems for the Uzbek language is complicated by low resource availability and the lack of standard tools. This paper presents a comparative analysis of four different named entity annotation schemes (BIO, BIOES, IO, and BILOU) in the context of Uzbek legislative texts. To assess the impact of annotation on the quality of entity extraction, a corpus of 3,000 sentences in four copies was formed. Each copy was annotated using a separate scheme. Based on the obtained data, four instances of the mBERT model were trained, and their performance was assessed using standard metrics (Precision, Recall, F1-score). The results showed that the choice of annotation scheme significantly affects the quality of the resulting NER models. The best results were achieved using BIOES (∼94% F1-score) and BILOU (∼92% F1-score), while BIO-scheme demonstrated intermediate quality (∼81% F1-score) and IO-scheme was the least effective (∼75% F1-score). Thus, in the context of a custom corpus, more detailed annotation schemes that allow explicit identification of entity boundaries are the preferred solution. This study highlights the importance of careful choice of annotation scheme when developing NER systems for Uzbek texts and provides a basis for further improvements in named entity recognition quality.

Topic ModelingNatural Language Processing TechniquesArtificial Intelligence in Lawnamed entity recognitionannotation schemeUzbek language

More by Nilufar Abdurakhmonova