Optimal adaptive control of methane production in anaerobic digestion via reinforcement learning

Abstract

This paper investigates the optimal regulation of methane production in a two-step anaerobic digestion (AD) process described by the AM2 model, incorporating Monod kinetics for acidogenesis and Haldane kinetics for methanogenesis. Following steady-state analysis, the nonlinear model is linearized around multiple equilibrium points, including both stable and unstable operating conditions, to obtain local state-space representations for control design. An infinite-horizon linear quadratic regulator (LQR) problem is formulated and solved online using a reinforcement learning approach based on policy iteration. The proposed method estimates the solution of the Riccati equation directly from measured state trajectories, without explicitly using the state matrix during the policy-evaluation step. The approach is applied using two manipulated inputs, namely the dilution rate and the inlet concentration of the second substrate. Numerical simulations evaluate the controller under multiple scenarios, including regulation around different equilibria, comparison with PI and predictive control strategies, and robustness to disturbances. The results indicate that the proposed method achieves local stabilization and improved regulation performance in simulation while reducing explicit reliance on the state matrix during policy evaluation.

More by Dr. Zarina Oflaz

• Development of machine learning models for intelligent prediction of workability, mechanical, and durability characteristics of cement mortar made of recycled sandConstruction and Building Materials · 2026 · Citations: 1
• Leveraging eco-friendly pozzolans and machine learning for reduced cement use in soil improvement of road base layersEuro-Mediterranean Journal for Environmental Integration · 2026
• Comparative Analysis of Prediction Accuracy in Clean Energy Forecasting: Implications for Pricing and Profitability in Wind and Biogas Energy SystemsSpringer proceedings in earth and environmental sciences · 2026
• The impact of changes in breast density over time on breast cancer riskScientific Reports · 2025 · Citations: 3