KIT | KIT-Bibliothek | Impressum | Datenschutz

Non-parametric policy search with limited information loss

Hoof, Herke van; Neumann, Gerhard; Peters, Jan


Learning complex control policies from non-linear and redundant sensory input is an important challenge for reinforcement learning algorithms. Non-parametric methods that approximate values functions or transition models can address this problem, by adapting to the complexity of the data set. Yet, many current non-parametric approaches rely on unstable greedy maximization of approximate value functions, which might lead to poor convergence or oscillations in the policy update. A more robust policy update can be obtained by limiting the information loss between successive state-action distributions. In this paper, we develop a policy search algorithm with policy updates that are both robust and non-parametric. Our method can learn non- parametric control policies for infinite horizon continuous Markov decision processes with non-linear and redundant sensory representations. We investigate how we can use approximations of the kernel function to reduce the time requirements of the demanding non-parametric computations. In our experiments, we show the strong performance of the proposed method, and how it can be approximated efficiently. Finally, we show that our algorithm can learn a real-robot under-powered swing-up task directly from image data.

Verlagsausgabe §
DOI: 10.5445/IR/1000118272
Veröffentlicht am 20.04.2020
Zitationen: 13
Web of Science
Zitationen: 9
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Anthropomatik und Robotik (IAR)
Publikationstyp Zeitschriftenaufsatz
Publikationsjahr 2017
Sprache Englisch
Identifikator ISSN: 1532-4435, 1533-7928
KITopen-ID: 1000118272
Erschienen in Journal of machine learning research
Verlag Journal of Machine Learning Research
Band 18
Heft 73
Seiten 1–46
Externe Relationen Abstract/Volltext
Schlagwörter Reinforcement Learning, Kernel Methods, Policy Search, Robotics
Nachgewiesen in Scopus
Web of Science
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page