Practical Approaches to Robotic Learning: Safety, Efficiency, and Adaptability

Humans possess a remarkable capacity to acquire new skills rapidly while maintaining safety throughout exploration. A child learning to ride a bicycle experiments with different balancing strategies and speeds, yet instinctively avoids behaviors that risk serious injury. This integration of safety into learning poses a central challenge in developing autonomous robotic agents that must operate in the physical world.

Reinforcement learning has achieved impressive results in simulated environments, yet deploying these systems on physical robots exposes critical gaps. Real-world robotics demands three essential capabilities: learning from limited interaction data, adapting quickly to new conditions, and maintaining safety throughout exploration. Current approaches address these requirements separately rather than as an integrated problem.

Model-free reinforcement learning offers a compelling advantage by learning control policies directly from experience without requiring accurate dynamics models. However, the approach demands extensive interaction, making it impractical for physical robots. Model-based methods address this sample-efficiency problem by learning predictive models that enable planning and rollouts in imagination, thereby reducing real-world data requirements. ... mehrThe tradeoff is that model inaccuracies can bias policy updates and degrade performance.

Beyond this model-based versus model-free distinction, two further research directions pursue complementary objectives. Safe reinforcement learning provides frameworks for constraint satisfaction, ensuring agents remain within safety limits during training. These methods, however, typically assume data abundance that physical systems cannot provide. Meta-learning focuses on rapid adaptation when encountering novel tasks or environments, but most existing frameworks ignore safety constraints and therefore expose the system to risk during adaptation.

The thesis develops four technical contributions. Constrained Model-Based Policy Optimization integrates learned dynamics with principled constraint enforcement, enabling sample-efficient policy learning under safety requirements. Model-Based Meta-Adaptation with Behavioral Constraints addresses the problem of adapting to novel dynamics by constraining the policy to remain close to safe reference behaviors from prior experience. Constrained Model-Agnostic Meta-Learning introduces a meta safety critic that guides meta-optimization toward initializations supporting both rapid adaptation and consistent constraint satisfaction. A fourth contribution tackles the computational challenges of vision-based policy learning by combining off-policy reinforcement learning with teacher-student distillation in massively parallel simulation for quadrupedal locomotion.

Extensive simulated experiments validate these contributions, and deployment of a vision-based locomotion policy on a Unitree GO2 robot demonstrates real-world applicability. The results show that treating safety as a foundational design principle rather than an auxiliary component yields algorithms that are simultaneously safe, sample-efficient, and rapidly adaptive.