UNIVERSAL TRANSFORMER FRAMEWORK FOR TEXT ANALYSIS AND GENERATION

Abstract

The article proposes a universal multimodal framework for solving three tasks of natural language processing: sarcasm recognition, sentiment-oriented generation of image descriptions, and contextual neural network machine translation using the reranking mechanism. The architecture includes specialized encoders (BERT, RoBERTa, ViT), cross-modal attention mechanisms, and contrastive learning, which provides adaptation to various types of input data and semantic tasks. The experiments demonstrated improvements in the BLEU, METEOR, F1, and NER metrics compared to the basic models. Special attention is paid to the stability of the model when working with rare words and named entities. The results obtained confirm the effectiveness of the proposed approach in conditions of limited data and multimodal complexity. In the context of the rapid growth of multimedia content and user expression on social networks, multimodal models combining text, visual and stylistic features are becoming a key direction in the development of cognitive AI systems. In particular, transformer architectures open up new horizons in the integration of multi-channel information, allowing us to take into account not only the direct meaning of the text, but also the emotional subtext, visual context and stylistic features of the presentation.

More by Ainur Akhmediyarova

• A Knowledge-Augmented Multi-Agent AI System for Context-Aware Legal Reasoning and Case Outcome PredictionPreprints.org · 2026
• Building a Multilingual AI Legal Assistant Using Retrieval-Augmented Generation: A Case Study on the Legal System of KazakhstanComputers · 2026
• Secure Federated Intrusion Detection for Resource-Constrained IoT Devices Using Lightweight Cryptography: A Hardware-Validated StudyFuture Internet · 2026 · Citations: 2
• Behavioural Biometrics and Session-Level Risk Monitoring for Insider Threat Detection in Enterprise NetworksElectronics · 2026