KIT | KIT-Bibliothek | Impressum | Datenschutz

Deep Reinforcement Learning for Swarm Systems

Hüttenrauch, Maximilian; Adrian, Sosic; Neumann, Gerhard

Abstract:

Recently, deep reinforcement learning (RL) methods have been applied successfully to multi-agent scenarios. Typically, the observation vector for decentralized decision making is represented by a concatenation of the (local) information an agent gathers about other agents. However, concatenation scales poorly to swarm systems with a large number of homogeneous agents as it does not exploit the fundamental properties inherent to these systems: (i) the agents in the swarm are interchangeable and (ii) the exact number of agents in the swarm is irrelevant. Therefore, we propose a new state representation for deep multi-agent RL based on mean embeddings of distributions, where we treat the agents as samples and use the empirical mean embedding as input for a decentralized policy. We define different feature spaces of the mean embedding using histograms, radial basis functions and neural networks trained end-to-end. We evaluate the representation on two well-known problems from the swarm literature — rendezvous and pursuit evasion — in a globally and locally observable setup. For the local setup we furthermore introduce simple communication protocols. ... mehr


Verlagsausgabe §
DOI: 10.5445/IR/1000118251
Veröffentlicht am 20.04.2020
Scopus
Zitationen: 123
Web of Science
Zitationen: 92
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Anthropomatik und Robotik (IAR)
Publikationstyp Zeitschriftenaufsatz
Publikationsjahr 2019
Sprache Englisch
Identifikator ISSN: 1532-4435, 1533-7928
KITopen-ID: 1000118251
Erschienen in Journal of machine learning research
Verlag Journal of Machine Learning Research
Band 20
Heft 54
Seiten 1–31
Externe Relationen Abstract/Volltext
Schlagwörter deep reinforcement learning, swarm systems, mean embeddings, neural networks, multi-agent systems
Nachgewiesen in Web of Science
Scopus
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page