KIT | KIT-Bibliothek | Impressum | Datenschutz

Sampling-Based Inverse Reinforcement Learning Algorithms with Safety Constraints

Fischer, Johannes ORCID iD icon; Eyberg, Christoph; Werling, Moritz; Lauer, Martin ORCID iD icon


Planning for robotic systems is frequently formulated as an optimization problem. Instead of manually tweaking the parameters of the cost function, they can be learned from human demonstrations by Inverse Reinforcement Learning (IRL). Common IRL approaches employ a maximum entropy trajectory distribution that can be learned with soft reinforcement learning, where the reward maximization is regularized with an entropy objective. The consideration of safety constraints is of paramount importance for human-robot collaboration. For this reason, our work addresses maximum entropy IRL in constrained environments. Our contribution to this research area is threefold: (1) We propose Constrained Soft Reinforcement Learning (CSRL), an extension of soft reinforcement learning to Constrained Markov Decision Processes (CMDPs). (2) We transfer maximum entropy IRL to CMDPs based on CSRL. (3) We show that using importance sampling in maximum entropy IRL in constrained environments introduces a bias and fails to achieve feature matching. In our evaluation we consider the tactical lane change decision of an autonomous vehicle in a highway scenario modeled in the SUMO traffic simulation.

DOI: 10.1109/IROS51168.2021.9636672
Zitationen: 5
Zitationen: 6
Zugehörige Institution(en) am KIT Institut für Mess- und Regelungstechnik (MRT)
Institut für Mess- und Regelungstechnik mit Maschinenlaboratorium (MRT)
Publikationstyp Proceedingsbeitrag
Publikationsdatum 16.12.2021
Sprache Englisch
Identifikator ISBN: 978-1-6654-1714-3
ISSN: 2153-0866
KITopen-ID: 1000136613
Erschienen in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prag, CZ, September 27 - October 1, 2021
Veranstaltung IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021), Prag, Tschechien, 27.09.2021 – 01.10.2021
Verlag Institute of Electrical and Electronics Engineers (IEEE)
Seiten 791 – 798
Nachgewiesen in Dimensions
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page