Learning Reward Machines: A Study In Partially Observable Reinforcement Learning

Abstract

Reinforcement learning (RL) is a central problem in artificial intelligence. This problem consists of defining artificial agents that can learn optimal behaviour by interacting with an environment -- where the optimal behaviour is defined with respect to a reward signal that the agent seeks to maximize. Reward machines (RMs) provide a structured, automata-based representation of a reward function that enables an RL agent to decompose an RL problem into structured subproblems that can be efficiently learned via off-policy learning. Here we show that RMs can be learned from experience, instead of being specified by the user, and that the resulting problem decomposition can be used to effectively solve partially observable RL problems. We pose the task of learning RMs as a discrete optimization problem where the objective is to find an RM that decomposes the problem into a set of subproblems such that the combination of their optimal memoryless policies is an optimal policy for the origin

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyicarte2021learning

Related papers

• Joint Learning Of Reward Machines And Policies In Environments With Partially Known Semantics (2022)3.58
• Learning Robust Reward Machines From Noisy Labels (2024)0.00
• Inferring Probabilistic Reward Machines From Non-markovian Reward Processes For Reinforcement Learning (2021)0.00
• Decentralized Graph-based Multi-agent Reinforcement Learning Using Reward Machines (2021)0.00
• Learning Task Automata For Reinforcement Learning Using Hidden Markov Models (2022)2.26
• A Hierarchical Bayesian Approach To Inverse Reinforcement Learning With Symbolic Reward Machines (2022)0.00
• Reinforcement Learning With Reward Machines In Stochastic Games (2023)0.00
• Contextual Pre-planning On Reward Machine Abstractions For Enhanced Transfer In Deep Reinforcement Learning (2023)2.26