A Markov Decision Model for a Surveillance Application and Risk-Sensitive Markov Decision Processes

First, we model the dynamics of threat of an infrastructure as a continuous-time Markov decision process. Several structural results are derived. Furthermore, the model is approximately solved by using an index-based heuristics.
Second, we consider discrete-time Markov decision processes. The criterion is to minimize the average value-at-risk for the discounted cost over a finite and over an infinite horizon. Moreover, the average value-at-risk criterion is considered for the average cost.