Talk · Journal of Physics Conference Series · January 1, 2021
Parallel news clustering and topic modeling approaches
Adai Shomanov, Мадина Мансурова
Abstract
Abstract At the current age there is an urgent need in developing massively scalable and efficient tools to Big Data processing. Even the smallest companies nowadays inevitably require more and more resources for data processing routines that could enhance decision making and reliably predict and simulate different scenarios. In the current paper we present our combined work on different massively scalable approaches for the task of clustering and topic modeling of the dataset, collected by crawling Kazakhstan news websites. In particular, we propose Apache Spark parallel solutions to news clustering and topic modeling problems and, additionally, we describe results of implementing document clustering using developed partitioned global address space Mapreduce system. In our work we describe our experience in solving these problems and investigate the efficiency and scalability of the proposed solutions.
More by Adai Shomanov
- EEG-XR: A Cross-Reality EEG Dataset for Neuroergonomic Analysis Across Desktop, Virtual, and Mixed RealityMendeley Data · 2026
- Next-generation BCI spellers: a comparative study of ERP responses and fatigue in mixed, virtual, and desktop realitiesVirtual Reality · 2025 · Citations: 2
- Enhancing Question Answering for Low-Resource Languages: The Case of Kazakh LanguageIEEE/ACM International Conference on Utility and Cloud Computing · 2025 · Citations: 1
- A Systematic Evaluation of Large Language Models and Retrieval-Augmented Generation for the Task of Kazakh Question AnsweringInformation · 2025 · Citations: 18