Reinforcement Learning for Controlled Traffic Rule Exceptions

Autonomous vehicles have the potential to revolutionize modern transportation systems. However, ensuring the safe and efficient operation of autonomous vehicles in complex traffic environments, especially those in traffic rule exception scenarios, is still a challenge. This thesis presents a novel approach to enhance the motion planning of autonomous vehicles in anomaly traffic scenarios through the integration of Deep Reinforcement Learning (DRL) with a structured rulebook.

The research begins by identifying the challenges faced by autonomous vehicles in coping with traffic rule exception scenarios, where traffic rules may differ from standard conditions. It then proceeds to present an in-depth literature review to gain insights into the current methods and traffic scenarios in motion planning and their limitations.

A method is then proposed, which leverages DreamerV3, a state-of-the-art DRL algorithm, to train an autonomous vehicle's driving policy. The method integrates trajectory generation as DRL output and a structured rulebook as part of the reward function of the DRL algorithm. The structured rulebook aims to encode the rules in a way that reflects the priority between rules, while its integration aims to improve the ability of agents to comply with these rules, while also allowing for transient rule violations in traffic rule exception scenarios by reflecting the priority between rules.
... mehr

The proposed method was rigorously tested using the CARLA simulation environment. Multiple training scenarios with varying complexity were designed to evaluate the effectiveness and robustness of the method in handling different anomaly traffic rule exception scenarios. The experimental results demonstrate that the proposed method outperforms the traditional control command-based methods and DRL methods without rulebook integration, in terms of both the learning curve and the final performance.

Though the research shows promising results, it is acknowledged that there are limitations regarding the scalability of the rulebook integration and the effectiveness of the method at higher speeds. Suggestions for future work include refining the rulebook integration process, investigating its effectiveness in high-speed scenarios, and exploring ways to automate the tuning of custom coefficients in the reward function.

In summary, this master thesis contributes to the field of autonomous driving by proposing a method that combines DRL with structured rulebook, thereby improving the performance of autonomous vehicles in traffic rule exception scenarios. The research provides valuable insights and sets a foundation for future work in the development of more robust and efficient autonomous driving systems.