KIT | KIT-Bibliothek | Impressum | Datenschutz

More Risk-Sensitive Markov Decision Processes

Bäuerle, Nicole ORCID iD icon 1; Rieder, Ulrich
1 Karlsruher Institut für Technologie (KIT)

Abstract:

We investigate the problem of minimizing a certainty equivalent of the total or discounted cost over a finite and an infinite horizon which is generated by a Markov Decision Process (MDP). The certainty equivalent is defined by U1(EU(Y )) where U is an increasing function. In contrast to a risk-neutral decision maker this optimization criterion takes the variability of the cost into account. It contains as a special case the classical risk-sensitive optimization criterion with an exponential utility. We show that this optimization problem can be solved by an ordinary MDP with extended state space and give conditions under which an optimal policy exists. In the case of an infinite time horizon we show that the minimal discounted cost can be obtained by value iteration and can be characterized as the unique
solution of a fixed point equation using a 'sandwich' argument. Interestingly, it turns out that in case of a power utility, the problem simplifies and is of similar complexity than the exponential utility case, however has not been treated in the literature so far. We also establish the validity (and convergence) of the policy improvement method. ... mehr


Volltext §
DOI: 10.5445/IR/1000039663
Originalveröffentlichung
DOI: 10.1287/moor.2013.0601
Scopus
Zitationen: 110
Web of Science
Zitationen: 101
Dimensions
Zitationen: 114
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Stochastik (STOCH)
Publikationstyp Zeitschriftenaufsatz
Publikationsjahr 2014
Sprache Englisch
Identifikator ISSN: 0364-765X
urn:nbn:de:swb:90-396638
KITopen-ID: 1000039663
Erschienen in Mathematics of operations research
Verlag Institute for Operations Research and Management Sciences (INFORMS)
Band 39
Heft 1
Seiten 105 - 120
Schlagwörter Markov Decision Problem, Certainty Equivalent, Positive Homogeneous, Utility, Exponential Utility, Value Iteration, Policy Improvement, Risk-sensitive Average Cost.
Nachgewiesen in Scopus
Dimensions
Web of Science
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page