KIT | KIT-Bibliothek | Impressum | Datenschutz

Stochastic Dynamic Programming with Non-linear Discounting

Bäuerle, Nicole; Jaśkiewicz, Anna; Nowak, Andrzej S.

In this paper, we study a Markov decision process with a non-linear discount function and with a Borel state space. We define a recursive discounted utility, which resembles non-additive utility functions considered in a number of models in economics. Non-additivity here follows from non-linearity of the discount function. Our study is complementary to the work of Jaśkiewicz et al. (Math Oper Res 38:108–121, 2013), where also non-linear discounting is used in the stochastic setting, but the expectation of utilities aggregated on the space of all histories of the process is applied leading to a non-stationary dynamic programming model. Our aim is to prove that in the recursive discounted utility case the Bellman equation has a solution and there exists an optimal stationary policy for the problem in the infinite time horizon. Our approach includes two cases: (a) when the one-stage utility is bounded on both sides by a weight function multiplied by some positive and negative constants, and (b) when the one-stage utility is unbounded from below.

Open Access Logo

Verlagsausgabe §
DOI: 10.5445/IR/1000129381
Veröffentlicht am 05.02.2021
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Stochastik (STOCH)
Publikationstyp Zeitschriftenaufsatz
Publikationsjahr 2020
Sprache Englisch
Identifikator ISSN: 0095-4616, 1432-0606
KITopen-ID: 1000129381
Erschienen in Applied mathematics & optimization
Verlag Springer
Schlagwörter Stochastic dynamic programming; Non-linear discounting; Bellman equation; Optimal stationary policy
Nachgewiesen in Scopus
Web of Science
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page