Comparative Evaluation of Evolutionary Hyperparameter Optimization for Gradient Boosting Ensembles in Clinical Multi-Class Classification

Abstract

In this study, the hyperparameters of Stochastic Gradient Boosting, LightGBM, and Histogram-based Gradient Boosting models were optimized using evolutionary algorithms. The main goal of this research is to find the most effective combination of hyperparameters for each model. Their goal is to increase accuracy and improve computational efficiency. The study was conducted on the basis of real clinical data. The data were obtained from the Samarkand City Endocrinology Center, using a dataset of diabetes-related and clinical indicators. The data were initially cleaned and processed using normalization, imputation, and 5-point cross-validation. We used five evolutionary strategies to fine-tune the main hyperparameters: Differential Evolution (DE), Genetic Algorithm (GA), Particle Swarm Optimization (PSO), Covariance Matrix Adaptation Evolution Strategy (CMA-ES), and Dynamic Ensemble with Hyperband (DEHB).In the optimization, the F1-macro was chosen as the main fitness function. This allows for a balanced accuracy assessment for multi-class classification. Across all three prototypes, evolutionary tuning consistently led to better results than Grid Search and Random Search. SGB+DE accuracy 0.9098, F1-macro 0.9096 gave the best results for the single model, whereas DEHB and CMA-ES worked better for LightGBM and HGB, respectively. The absolute increases above the unadjusted baselines ranged from 0.3 to 0.6 percentage points. They were small but could be repeated reliably across all evaluation criteria. During this study, the models were evaluated on the metrics of accuracy, precision, recall, and F1-macro.

More by Mekhriddin Nurmamatov

• Lean Quantum Intelligence: A Compact Variational-Circuit Hybrid for Turbofan RUL Prediction with SHAP AttributionComputers · 2026
• Investigation of Vulnerabilities of Machine Learning Models to Adversarial Attacks in Smart Grid Forecasting SystemsAdvances in computational intelligence and robotics book series · 2026
• Capacity-Constrained Multi-robot Task Allocation via Quantum Approximate Optimisation: A Reproducible Benchmark on a Solomon VRPTW C101-Derived InstanceCommunications in computer and information science · 2026
• Forecasting Electrical Load and Renewable Energy Generation Using Machine Learning MethodsAdvances in computational intelligence and robotics book series · 2026