We investigate the problem of minimizing a certainty equivalent of the total or discounted cost over a finite and an infinite horizon which is generated by a Markov Decision Process (MDP). The certainty equivalent is defined by U1(EU(Y )) where U is an increasing function. In contrast to a risk-neutral decision maker this optimization criterion takes the variability of the cost into account. It contains as a special case the classical risk-sensitive optimization criterion with an exponential utility. We show that this optimization problem can be solved by an ordinary MDP with extended state space and give conditions under which an optimal policy exists. In the case of an infinite time horizon we show that the minimal discounted cost can be obtained by value iteration and can be characterized as the unique

solution of a fixed point equation using a 'sandwich' argument. Interestingly, it turns out that in case of a power utility, the problem simplifies and is of similar complexity than the exponential utility case, however has not been treated in the literature so far. We also establish the validity (and convergence) of the policy im ... mehr

solution of a fixed point equation using a 'sandwich' argument. Interestingly, it turns out that in case of a power utility, the problem simplifies and is of similar complexity than the exponential utility case, however has not been treated in the literature so far. We also establish the validity (and convergence) of the policy im ... mehr

Zugehörige Institution(en) am KIT |
Institut für Stochastik (STOCH) |

Publikationstyp |
Zeitschriftenaufsatz |

Jahr |
2014 |

Sprache |
Englisch |

Identifikator |
ISSN: 0364-765X KITopen ID: 1000039663 |

Erschienen in |
Mathematics of operations research |

Band |
39 |

Heft |
1 |

Seiten |
105 - 120 |

Schlagworte |
Markov Decision Problem, Certainty Equivalent, Positive Homogeneous Utility, Exponential Utility, Value Iteration, Policy Improvement, Risk-sensitive Average Cost. |

KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page