Talk · International Conference on Electronics, Engineering Physics and Earth Science · September 10, 2026
Analysis of Multimodal Heterogeneous Graph Transformers for Reconstructing Social Network Structures Based on Open-Source Data
Azat Abutalifov, Anel Aidos, Assel Mukasheva, Alibek Bissembayev, Мадина Сыдыбаева, Arailym Keneskanova
Abstract
In this paper, transformer-based graph neural networks are applied across different data types, such as text, images, and metadata, to automatically infer the structure of social networks from publicly available OSINT data. A pipeline was built that first processes text with BERT and images with ResNet, then combines them with an attention mechanism, and finally feeds the fused representations into a Heterogeneous Graph Transformer to classify nodes and predict links between them. Experiments were conducted on two publicly available Twitter datasets for research: Cresci-15 and Twibot-20. For node classification, our method achieved F1-scores of 0.979 and 0.882 on these datasets, which are better than using only text or only the graph structure. For link prediction, we got an AUC of 0.871 on Twibot-20. Combining different types of data with a heterogeneous graph structure performs better than using a single type, but our method has certain limitations that warrant future work.
More by Assel Mukasheva
- Performance Evaluation of WPA2 and WPA3 Security Protocols in Private Wi-Fi 6 Environments2026
- HYBRID GAT-AUTOENCODER-BILSTM-ATTENTION ARCHITECTURE FOR MULTI CLASS CYBERATTACK DETECTION IN HETEROGENEOUS IOT NETWORKSВестник Алматинского университета энергетики и связи · 2026
- DISTRIBUTED SYSTEM FOR DATA SHARING AND JOINT PROCESSING BETWEEN RESEARCH CENTERS AND FARMSВестник Алматинского университета энергетики и связи · 2026
- Comparative Study of Regression Models Using Machine Learning to Predict Real Stock PricesInternational Conference on Electronics, Engineering Physics and Earth Science · 2026