Talk · International Conference on Computer Science and Engineering · September 14, 2022
Applying Web Crawler Technologies for Compiling Parallel Corpora as one Stage of Natural Language Processing
Nilufar Z. Abdurakhmonova, Ismailov Alisher, Guli Ibragimovna Toirova
Abstract
over the past decade, the amount of information on the internet has increased. A large amount of unstructured data, referred to as big data on the web, has been created. Finding and extracting data on the internet is called information retrieval. In the search for information, there are web crawler tools, which are a program that scans information on the internet and downloads web documents automatically. Search robot applications can be used in various fields, such as news, finance, medicine, etc. In this article, we will discuss the basic principle and characteristics of search engines as an example to build parallel corpora, as well as the classification of modern popular crawlers, strategies and current applications of crawlers. Finally, we will end this article with a discussion of future directions for research on crawlers.
More by Nilufar Abdurakhmonova
- A model of a universal linguistic knowledge graph for Turkic languagesArctic XXI century · 2026
- Machine Learning-Based Topic Classification of Economic and Financial Texts in UzbekInternational Conference on Computer Science and Engineering · 2026
- Building a Phraseological Ontology of Uzbek Idioms and Evaluating Similarity Measures for Their Detection in TextInternational Conference on Computer Science and Engineering · 2026
- Developing Morphological and Syntactic Analysis Models for Karakalpak as a Low-Resource Turkic LanguageInternational Conference on Computer Science and Engineering · 2026