On The Power Of Pre-training For Generalization In RL: Provable Benefits And Hardness

Abstract

Generalization in Reinforcement Learning (RL) aims to learn an agent during training that generalizes to the target environment. This paper studies RL generalization from a theoretical aspect: how much can we expect pre-training over training environments to be helpful? When the interaction with the target environment is not allowed, we certify that the best we can obtain is a near-optimal policy in an average sense, and we design an algorithm that achieves this goal. Furthermore, when the agent is allowed to interact with the target environment, we give a surprising result showing that asymptotically, the improvement from pre-training is at most a constant factor. On the other hand, in the non-asymptotic regime, we design an efficient algorithm and prove a distribution-based regret bound in the target environment that is independent of the state-action space.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyye2022on

Related papers

• The Principle Of Unchanged Optimality In Reinforcement Learning Generalization (2019)0.00
• Assessing Generalization In Deep Reinforcement Learning (2018)0.00
• The Benefits Of Model-based Generalization In Reinforcement Learning (2022)0.00
• Good Actions Succeed, Bad Actions Generalize: A Case Study On Why RL Generalizes Better (2025)0.00
• Dynamics Generalization Via Information Bottleneck In Deep Reinforcement Learning (2020)0.00
• Illuminating Generalization In Deep Reinforcement Learning Through Procedural Level Generation (2018)0.00
• Measuring And Characterizing Generalization In Deep Reinforcement Learning (2018)9.76
• Quantifying Generalization In Reinforcement Learning (2018)0.00