In this paper we consider an explicit dynamic risk measure for discrete-time payment processes which have a Markovian structure. The risk measure is essentially a sum of conditional Average Value-at-Risks. Analogous to the static Average Value-at-Risk, this risk measures can be reformulated in terms of the value functions of a dynamic optimization problem, namely a so-called Markov decision problem. This observation gives a nice recursive computation formula. Afterwards, the definition of the dynamic risk measure is generalized to a setting with incomplete information about the risk distribution which can be seen as model ambiguity. We choose a parametric approach here. The dynamic risk measure is again defined as the sum of conditional Average Value-at-Risks or equivalently is the solution of a Bayesian decision problem. Finally, it is possible to discuss the effect of model ambiguity on the risk measure: Surprisingly, it may be the case that the risk decreases when additional "risk" due to parameter uncertainty shows up. All investigations are illustrated by a simple but useful coin tossing game proposed by Artzner and b ... mehry the classical Cox-Ross-Rubinstein model.