Look Where You Look! Saliency-guided Q-networks For Generalization In Visual Reinforcement Learning

Abstract

Deep reinforcement learning policies, despite their outstanding efficiency in simulated visual control tasks, have shown disappointing ability to generalize across disturbances in the input training images. Changes in image statistics or distracting background elements are pitfalls that prevent generalization and real-world applicability of such control policies. We elaborate on the intuition that a good visual policy should be able to identify which pixels are important for its decision, and preserve this identification of important sources of information across images. This implies that training of a policy with small generalization gap should focus on such important pixels and ignore the others. This leads to the introduction of saliency-guided Q-networks (SGQN), a generic method for visual reinforcement learning, that is compatible with any value function learning method. SGQN vastly improves the generalization capability of Soft Actor-Critic agents and outperforms existing stateof

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybertoin2022look

Related papers

• Rethinking Value Function Learning For Generalization In Reinforcement Learning (2022)0.00
• Are Gradient-based Saliency Maps Useful In Deep Reinforcement Learning? (2020)0.00
• Understanding What Affects The Generalization Gap In Visual Reinforcement Learning: Theory And Empirical Evidence (2024)5.84
• Measuring And Characterizing Generalization In Deep Reinforcement Learning (2018)9.76
• Robust Visual Domain Randomization For Reinforcement Learning (2019)0.00
• A Recipe For Unbounded Data Augmentation In Visual Reinforcement Learning (2024)0.00
• Quantum Natural Policy Gradients: Towards Sample-efficient Reinforcement Learning (2023)7.16
• Natural Gradient Deep Q-learning (2018)0.00