Skip to content
DSML.KZ
Publications

Review · International Conference on Information and Network Technologies · March 6, 2026

Federated Learning Strategies for Fine-Tuning Large Language Models: A Systematic Literature Review

Adil Akhmetov, Mohammed Alaa Ala’anzy, Amani Ibraheem

Mohammed Alaa Fadhil Al-hadeethi
Open via DOI

Abstract

Recently, the advancement in natural language processing has been driven by the concept of Large Language Models (LLMs). Nevertheless, the deployment of these models in distributed systems is associated with an essential trade-off in terms of privacy and model performance. This systematic review examines the various methods of federated learning in fine-tuning LLMs that were developed between 2023 and 2025, considering the trade-offs in terms of privacy and performance. The results indicate that the methods of Parameter-Efficient Fine-Tuning (PEFT) have been successful in reducing the bandwidth requirements, while system heterogeneity is the key challenge. In the near future, the research in the area of fine-tuning LLMs needs to be focused on architectural flexibility to suit the constraints of edge devices.

Privacy-Preserving Technologies in DataDomain Adaptation and Few-Shot LearningTopic Modelingfederated learninglarge language modelsparameter-efficient fine-tuning

More by Mohammed Alaa Fadhil Al-hadeethi