KIT | KIT-Bibliothek | Impressum | Datenschutz
Open Access Logo
URN: urn:nbn:de:swb:90-396638
DOI: 10.1287/moor.2013.0601
Zitationen: 24
Web of Science
Zitationen: 18

More Risk-Sensitive Markov Decision Processes

Bäuerle, Nicole; Rieder, Ulrich

We investigate the problem of minimizing a certainty equivalent of the total or discounted cost over a finite and an infinite horizon which is generated by a Markov Decision Process (MDP). The certainty equivalent is defined by U1(EU(Y )) where U is an increasing function. In contrast to a risk-neutral decision maker this optimization criterion takes the variability of the cost into account. It contains as a special case the classical risk-sensitive optimization criterion with an exponential utility. We show that this optimization problem can be solved by an ordinary MDP with extended state space and give conditions under which an optimal policy exists. In the case of an infinite time horizon we show that the minimal discounted cost can be obtained by value iteration and can be characterized as the unique
solution of a fixed point equation using a 'sandwich' argument. Interestingly, it turns out that in case of a power utility, the problem simplifies and is of similar complexity than the exponential utility case, however has not been treated in the literature so far. We also establish the validity (and convergence) of the policy im ... mehr

Zugehörige Institution(en) am KIT Institut für Stochastik (STOCH)
Publikationstyp Zeitschriftenaufsatz
Jahr 2014
Sprache Englisch
Identifikator ISSN: 0364-765X
KITopen ID: 1000039663
Erschienen in Mathematics of operations research
Band 39
Heft 1
Seiten 105 - 120
Schlagworte Markov Decision Problem, Certainty Equivalent, Positive Homogeneous Utility, Exponential Utility, Value Iteration, Policy Improvement, Risk-sensitive Average Cost.
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft KITopen Landing Page