KIT | KIT-Bibliothek | Impressum | Datenschutz

Safety Aware Reinforcement Learning by Identifying Comprehensible Constraints in Expert Demonstrations

Müller, Leopold 1; Böcking, Lars 1; Färber, Michael ORCID iD icon 2
1 Karlsruher Institut für Technologie (KIT)
2 Institut für Angewandte Informatik und Formale Beschreibungsverfahren (AIFB), Karlsruher Institut für Technologie (KIT)

Abstract:

When used in real-world environments, agents must meet high safety requirements as errors have direct consequences. Besides the safety aspect, the explainability of the systems is of particular importance. Therefore, not only should errors be avoided during the learning process, but also the decision process should be made transparent. Existing approaches are limited to solving a single problem. For real-world use, however, several criteria must be fulfilled at the same time. In this paper we derive comprehensible rules from expert demonstrations which can be used to monitor the agent. The developed approach uses state of the art classification and regression trees for deriving safety rules combined with concepts in the field of association rule mining. The result is a compact and comprehensible rule set that explains the expert’s behavior and ensures safety. We evaluate our framework in common OpenAI environments. Results show that the elaborated approach is able to identify safety-relevant rules and imitate expert behavior especially in edge cases. Evaluations on higher dimensional observation spaces and continuous action spaces highlight the transferability of the approach to new tasks while maintaining compactness and comprehensibility of the rule set.


Verlagsausgabe §
DOI: 10.5445/IR/1000143668
Veröffentlicht am 11.03.2022
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Angewandte Informatik und Formale Beschreibungsverfahren (AIFB)
Publikationstyp Proceedingsbeitrag
Publikationsjahr 2022
Sprache Englisch
Identifikator ISSN: 1613-0073
KITopen-ID: 1000143668
Erschienen in Proceedings of the Workshop on Artificial Intelligence Safety 2022 (SafeAI 2022): co-located with the Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI2022) ; Virtual, February, 2022
Veranstaltung AAAI Workshop on Artificial Intelligence Safety (SafeAI 2022), Online, 28.02.2022 – 01.03.2022
Verlag CEUR-WS.org
Serie CEUR Workshop Proceedings ; 3087
Nachgewiesen in Scopus
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page