A Concept of Unbiased Stochastically Deterministic Policy Gradient for Better Convergence in Bipedal Walker

Abstract

The main goal of this work is to play a role of playground to further improve our own conceptual model, or “unbiased Deep Deterministic Policy Gradient (uDDPG)” developed recently: due to search for an algorithm that can provide faster convergence to better policies the agent can meet a problem of falling to the local optimum regions and slightly improve from there for the rest of experiments. In other words, proposed concept has a problem of limited policy generalization, especially for more complex systems. To achieve a better exploration and consequently search for higher scores, we tried to equip our concept with simplified ideas from the notable works: we unanchored uDDPG or in parallel added Q network's update without the link to real Q values but with Entropy Regularization of soft Q learning based algorithms, e.g. Soft Actor and Critic; additionally, reduced ideas behind Forward-Looking Actor (FORK), Twin Delayed DDPG and Distributional Perspective on Q value were utilized for more “optimistic” but “safer” search.

More by Zhenis Otarbay

• Ocular-correlated signals and decision rules in DTW decoding of motor-imagery EEG: code and resultsZenodo · 2026
• When Deployment Changes a Learned Action: Post-Inference Projection in Offline Multi-Agent Reinforcement LearningAI · 2026
• Robust Offline Multi-Agent Reinforcement Learning for Latency-Aware SDN Path Control in 6G-Oriented Network SoftwarizationTechnologies · 2026 · Citations: 1
• An Exploratory Study of Gender Effects on Pupillary Cognitive Load in Human–Robot InteractionIEEE International Conference on Smart Information Systems and Technologies · 2026