Learning To Identify Critical States For Reinforcement Learning From Videos

Abstract

Recent work on deep reinforcement learning (DRL) has pointed out that algorithmic information about good policies can be extracted from offline data which lack explicit information about executed actions. For example, videos of humans or robots may convey a lot of implicit information about rewarding action sequences, but a DRL machine that wants to profit from watching such videos must first learn by itself to identify and recognize relevant states/actions/rewards. Without relying on ground-truth annotations, our new method called Deep State Identifier learns to predict returns from episodes encoded as videos. Then it uses a kind of mask-based sensitivity analysis to extract/identify important critical states. Extensive experiments showcase our method's potential for understanding and improving agent behavior. The source code and the generated datasets are available at https://github.com/AI-Initiative-KAUST/VideoRLCS.

Stats

• citations5
• S2 citations—
• github stars28
• HF likes0
• heat score8.76
• arxiv keyliu2023learning

Code

• AI-Initiative-KAUST/VideoRLCS—

Related papers

• Diagnosing And Exploiting The Computational Demands Of Videos Games For Deep Reinforcement Learning (2023)0.00
• Value-consistent Representation Learning For Data-efficient Reinforcement Learning (2022)0.00
• A Survey Of Deep Reinforcement Learning In Video Games (2019)0.00
• Viva: Video-trained Value Functions For Guiding Online RL From Diverse Data (2025)0.00
• Visual Explanation Using Attention Mechanism In Actor-critic-based Deep Reinforcement Learning (2021)10.21
• Episodic Reinforcement Learning With Expanded State-reward Space (2024)0.00
• Learning From Demonstrations With SACR2: Soft Actor-critic With Reward Relabeling (2021)0.00
• Visual Processing In Context Of Reinforcement Learning (2022)0.00