KIT | KIT-Bibliothek | Impressum | Datenschutz

Mean Field Markov Decision Processes

Bäuerle, Nicole ORCID iD icon 1,2
1 Fakultät für Mathematik (MATH), Karlsruher Institut für Technologie (KIT)
2 Institut für Stochastik (STOCH), Karlsruher Institut für Technologie (KIT)

Abstract:

We consider mean-field control problems in discrete time with discounted reward, infinite time horizon and compact state and action space. The existence of optimal policies is shown and the limiting mean-field problem is derived when the number of individuals tends to infinity. Moreover, we consider the average reward problem and show that the optimal policy in this mean-field limit is ε-optimal for the discounted problem if the number of individuals is large and the discount factor close to one. This result is very helpful, because it turns out that in the special case when the reward does only depend on the distribution of the individuals, we obtain a very interesting subclass of problems where an average reward optimal policy can be obtained by first computing an optimal measure from a static optimization problem and then achieving it with Markov Chain Monte Carlo methods. We give two applications: Avoiding congestion an a graph and optimal positioning on a market place which we solve explicitly.


Verlagsausgabe §
DOI: 10.5445/IR/1000159695
Veröffentlicht am 26.06.2023
Originalveröffentlichung
DOI: 10.1007/s00245-023-09985-1
Scopus
Zitationen: 4
Web of Science
Zitationen: 2
Dimensions
Zitationen: 5
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Stochastik (STOCH)
Publikationstyp Zeitschriftenaufsatz
Publikationsmonat/-jahr 08.2023
Sprache Englisch
Identifikator ISSN: 0095-4616, 1432-0606
KITopen-ID: 1000159695
Erschienen in Applied Mathematics & Optimization
Verlag Springer
Band 88
Heft 1
Seiten Art.-Nr.: 12
Vorab online veröffentlicht am 10.04.2023
Nachgewiesen in Web of Science
Dimensions
Scopus
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page