Reinforcement Learning for Speed Control with Feedforward to Track Velocity Profiles in a Real Vehicle

Reinforcement learning (RL) for tracking arbitrary trajectories in real control applications with unobserved dynamics is a challenging subject of current research. An example of such an application is given by the speed control of a real car with unobserved longitudinal dynamics. We propose an online RL method, where the RL agent explicitly incorporates information concerning the desired velocity profile by means of a time-varying first order trajectory approximation. Furthermore, missing state information is reconstructed by learning a finite impulse response filter. We apply the presented algorithm in online training of controller parameters in a real car. Our validation results reveal that our proposed method clearly outperforms a controller that uses set points of the desired velocity by means of the resulting velocity tracking performance.