Article · Computers and Electronics in Agriculture · July 2, 2026

Robust multimodal fusion for terrain perception and autonomous robot navigation in unstructured agricultural environments

Chenghan Yang, Dingkun Zheng, Madina Mansurova, Baurzhan Belgibaev, G. A. Amirkhanova, Б. Чжао

Abstract

Autonomous navigation of agricultural robots faces persistent challenges in unstructured environments, including canopy occlusion, specular reflections from pooled water, and ridge-induced platform vibrations that cause temporally varying sensor degradation. Existing multimodal systems fail to maintain both reliability and real-time responsiveness, while accurate delineation of safety–critical classes (especially waterlogged zones) and the direct translation of perceptual quality into navigational safety remain key bottlenecks. We present a lightweight real-time adaptive multimodal perception system that establishes a complete perception–planning closed loop. Built on MobileNetV3-U-Net-Lite, the architecture integrates (1) six-channel early RGB–LiDAR fusion for deep cross-modal complementarity from the shallowest layers, (2) stepwise collaborative filtering with learnable anisotropic boundary weights supervised by a composite CE + Dice + boundary loss to handle directional terrain margins and low-frequency safety–critical classes, and (3) a 10-dimensional physical state vector (IMU vibration signatures + RGB illumination descriptors) that drives an MLP-based dynamic modality weighting mechanism reinforced by physical-consistency regularization. The fused output generates a quantitative risk map encoding slope, surface undulation, water-accumulation probability, and vegetation obscuration, which directly feeds the Agri Path planner for risk-guided path refinement and global map accumulation. Deployed on the NVIDIA Jetson Orin NX, the system achieves 47 FPS with 3.4 M parameters and 7.8 W power consumption. On the RELLIS-3D benchmark it attains 62.56 % mIoU, 40.42 % puddle IoU, and boundary F1 scores exceeding all baselines. Field trials at Xihua University demonstrate a 91.7 % improvement in minimum safe distance over the IMU-ablated baseline and reduction of peak yaw-rate to 37.5 % of the control group. These results demonstrate a viable pathway toward physically grounded, temporally adaptive safety navigation: shifting from static precision-centric fusion to physically grounded, temporally adaptive, risk-quantified safety navigation under tight edge-computing constraints. Field validation under dusk post-irrigation conditions confirms system robustness under compound sensor degradation; the proposed mechanisms demonstrate strong potential for transfer to orchards, vegetable fields, and wheat croplands, subject to further validation under full nocturnal and adverse-weather conditions.

Smart Agriculture and AIRobotics and Sensor-Based LocalizationAutonomous Vehicle Technology and Safetyagricultural roboticsautonomous navigationmultimodal sensor fusion

Robust multimodal fusion for terrain perception and autonomous robot navigation in unstructured agricultural environments

Abstract

More by Baurzhan Belgibaev