Learning Symbolic Representations For Reinforcement Learning Of Non-markovian Behavior

Abstract

Many real-world reinforcement learning (RL) problems necessitate learning complex, temporally extended behavior that may only receive reward signal when the behavior is completed. If the reward-worthy behavior is known, it can be specified in terms of a non-Markovian reward function - a function that depends on aspects of the state-action history, rather than just the current state and action. Such reward functions yield sparse rewards, necessitating an inordinate number of experiences to find a policy that captures the reward-worthy pattern of behavior. Recent work has leveraged Knowledge Representation (KR) to provide a symbolic abstraction of aspects of the state that summarize reward-relevant properties of the state-action history and support learning a Markovian decomposition of the problem in terms of an automaton over the KR. Providing such a decomposition has been shown to vastly improve learning rates, especially when coupled with algorithms that exploit automaton structure. N

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keychristoffersen2023learning

Related papers

• Learning Sparse Representations In Reinforcement Learning (2019)0.00
• Learning Temporally-consistent Representations For Data-efficient Reinforcement Learning (2021)0.00
• Learning Task Automata For Reinforcement Learning Using Hidden Markov Models (2022)2.26
• Reinforcement Learning With Knowledge Representation And Reasoning: A Brief Survey (2023)0.00
• Demystifying Reinforcement Learning In Time-varying Systems (2022)0.00
• Spectral Representation-based Reinforcement Learning (2025)0.00
• A First-occupancy Representation For Reinforcement Learning (2021)0.00
• A Hierarchical Bayesian Approach To Inverse Reinforcement Learning With Symbolic Reward Machines (2022)0.00