Skip to content
DSML.KZ
Publications

Article · Symmetry · August 4, 2026

Validation-Generalization Asymmetry in Metaheuristic CNN Forecasting: An Extremely Small Time-Series Case

Б. Чжао, Hao Yu, Laixing Jing, Aman Mussa, Chenghan Yang, Dingkun Zheng

Sholpan Jomartova
Open via DOI

Abstract

Hyperparameter optimization can reduce validation error while worsening future performance when data are extremely scarce. This study examines that risk in a public railway-mileage implementation workbook containing 17 ordered five-lag input-target rows, of which 14 form the development set and three form a locked outer test. No independently verified 22-point raw series was available; exact archived rows are therefore the primary data, while an overlap-consistent inferred chronology is used only for sensitivity analysis. WOA-CNN and BWO-CNN were evaluated through 10 complete HPO searches under shuffled and rolling-origin validation. Five-search purged runs were used as a stress-test sensitivity, while a fixed Manual CNN served as the matched reference. Median locked-test MAPE was 3.75% for WOA-Random, 2.56% for BWO-Random, 4.16% for WOA-Rolling, and 4.51% for BWO-Rolling, compared with a matched Manual CNN reference mean of 2.54%. The four HPO procedures beat that reference in only 3/10, 5/10, 2/10, and 1/10 searches, respectively. A mean-increment baseline obtained 0.63% MAPE. A separate benchmark of 160 independent simulated series examines sample-size and validation-selection mechanisms under four pre-specified data-generating processes. Taken together, the two analyses characterize how validation design and data availability affect model-selection stability in extremely small time-series settings.

Forecasting Techniques and ApplicationsMachine Learning and Data ClassificationMetaheuristic Optimization Algorithms Researchmetaheuristic hyperparameter optimizationdata scarcitywhale optimization algorithm

More by Sholpan Jomartova