KIT | KIT-Bibliothek | Impressum | Datenschutz

ConstrainedZero: Chance-Constrained POMDP Planning Using Learned Probabilistic Failure Surrogates and Adaptive Safety Constraints

Moss, Robert J.; Jamgochian, Arec; Fischer, Johannes ORCID iD icon 1; Corso, Anthony; Kochenderfer, Mykel J.
1 Institut für Mess- und Regelungstechnik (MRT), Karlsruher Institut für Technologie (KIT)

Abstract:

To plan safely in uncertain environments, agents must balance utility with safety constraints. Safe planning problems can be modeled as a chance-constrained partially observable Markov decision process (CC-POMDP) and solutions often use expensive rollouts or heuristics to estimate the optimal value and action-selection policy. This work introduces the ConstrainedZero policy iteration algorithm that solves CC-POMDPs in belief space by learning neural network approximations of the optimal value and policy with an additional network head that estimates the failure probability given a belief. This failure probability guides safe action selection during online Monte Carlo tree search (MCTS). To avoid overemphasizing search based on the failure estimates, we introduce Δ-MCTS, which uses adaptive conformal inference to update the failure threshold during planning. The approach is tested on a safety-critical POMDP benchmark, an aircraft collision avoidance system, and the sustainability problem of safe CO$_2$ storage. Results show that by separating safety constraints from the objective we can achieve a target level of safety without optimizing the balance between rewards and costs.


Volltext §
DOI: 10.5445/IR/1000175010
Veröffentlicht am 10.10.2024
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Mess- und Regelungstechnik (MRT)
Publikationstyp Forschungsbericht/Preprint
Publikationsjahr 2024
Sprache Englisch
Identifikator KITopen-ID: 1000175010
Umfang 10 S.
Vorab online veröffentlicht am 01.05.2024
Nachgewiesen in arXiv
Relationen in KITopen
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page