Open-ended Diverse Solution Discovery With Regulated Behavior Patterns For Cross-domain Adaptation

Abstract

While Reinforcement Learning can achieve impressive results for complex tasks, the learned policies are generally prone to fail in downstream tasks with even minor model mismatch or unexpected perturbations. Recent works have demonstrated that a policy population with diverse behavior characteristics can generalize to downstream environments with various discrepancies. However, such policies might result in catastrophic damage during the deployment in practical scenarios like real-world systems due to the unrestricted behaviors of trained policies. Furthermore, training diverse policies without regulation of the behavior can result in inadequate feasible policies for extrapolating to a wide range of test conditions with dynamics shifts. In this work, we aim to train diverse policies under the regularization of the behavior patterns. We motivate our paradigm by observing the inverse dynamics in the environment with partial state information and propose Diversity in Regulation (DiR) trai

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyxu2022open

Related papers

• Diversity For Contingency: Learning Diverse Behaviors For Efficient Adaptation And Transfer (2023)0.00
• Diversity-inducing Policy Gradient: Using Maximum Mean Discrepancy To Find A Set Of Diverse Policies (2019)8.35
• Effective Diversity In Population Based Reinforcement Learning (2020)0.00
• Learning A Subspace Of Policies For Online Adaptation In Reinforcement Learning (2021)0.00
• Diverse Exploration For Fast And Safe Policy Improvement (2018)4.52
• Diverse Policies Converge In Reward-free Markov Decision Processe (2023)0.00
• DGPO: Discovering Multiple Strategies With Diversity-guided Policy Optimization (2022)2.26
• Unifying Behavioral And Response Diversity For Open-ended Learning In Zero-sum Games (2021)0.00