The reinforcement learning for autonomous accelerators collaboration

Reinforcement Learning (RL) is a unique learning paradigm that is particularly well-suited to tackle complex control tasks, can deal with delayed consequences, and can learn from experience without an explicit model of the dynamics of the problem. These properties make RL methods extremely promising for applications in particle accelerators, where the dynamically evolving conditions of both the particle beam and the accelerator systems must be constantly considered. While the time to work on RL is now particularly favorable thanks to the availability of high-level programming libraries and resources, its implementation in particle accelerators is not trivial and requires further consideration. In this context, the Reinforcement Learning for Autonomous Accelerators (RL4AA) international collaboration was established to consolidate existing knowledge, share experiences and ideas, and collaborate on accelerator-specific solutions that leverage recent advances in RL. Here we report on two collaboration workshops, RL4AA'23 and RL4AA'24, which took place in February 2023 at the Karlsruhe Institute of Technology and in February 2024 at the Paris-Lodron Universität Salzburg.