Hierarchical reinforcement learning with multi-step actions

In recent years hierarchical concepts of temporal abstraction
have been integrated in the reinforcement learning framework to
improve scalability. However, existing approaches are limited to
domains for which a decomposition in subtasks is known a priori.
In this paper we propose the concept of multi-step actions on
different time scales in one single action set. It is suited for
learning optimal policies in unstructured domains where a
decomposition is not known in advance or does not exist at all.
At the same time this approach enables learning at multiple
levels of temporal abstraction. Thus, multi-step actions offer
the possibility to obtain faster learning algorithms for
unstructured domains.