Robust Parameter Estimation and Tracking through Lyapunov-based Actor-Critic Reinforcement Learning

This work presents an approach for parameter estimation of nonlinear systems by means of robust maximum entropy offline reinforcement learning (RL). The identification of parameter variant systems is a challenging problem in industrial applications. We address the parameter estimation on disturbed measurements with an actor-critic RL agent that is extended by a Lyapunov neural network. Accordingly, a robust soft actor-critic algorithm (RSAC) is applied to the parametrization problem. The policy is learned on test trajectories and can be applied to identify nonlinear system parameter maps for comparable system dynamics across an operating range. In a simulative study, the performance of the proposed concept is shown for a permanent magnet synchronous machine model in the d/q-frame formulation with current dependent and nonlinear flux linkages. The trained RL agent is evaluated on a different nonlinear machine parameter set under noisy measurements. The results indicate applicability to real-world tasks such as system parameter tracking.