Logical Specifications-guided Dynamic Task Sampling For Reinforcement Learning Agents

Abstract

Reinforcement Learning (RL) has made significant strides in enabling artificial agents to learn diverse behaviors. However, learning an effective policy often requires a large number of environment interactions. To mitigate sample complexity issues, recent approaches have used high-level task specifications, such as Linear Temporal Logic (LTL\(_f\)) formulas or Reward Machines (RM), to guide the learning progress of the agent. In this work, we propose a novel approach, called Logical Specifications-guided Dynamic Task Sampling (LSTS), that learns a set of RL policies to guide an agent from an initial state to a goal state based on a high-level task specification, while minimizing the number of environmental interactions. Unlike previous work, LSTS does not assume information about the environment dynamics or the Reward Machine, and dynamically samples promising tasks that lead to successful goal policies. We evaluate LSTS on a gridworld and show that it achieves improved time-to-thresh

Stats

• citations1
• S2 citations—
• github stars0
• HF likes0
• heat score2.26
• arxiv keyshukla2024logical

Related papers

• Sample-efficient Reinforcement Learning With Temporal Logic Objectives: Leveraging The Task Specification To Guide Exploration (2024)0.00
• Directed Exploration In Reinforcement Learning From Linear Temporal Logic (2024)0.00
• A Policy Search Method For Temporal Logic Specified Reinforcement Learning Tasks (2017)11.58
• Sample Efficient Model-free Reinforcement Learning From LTL Specifications With Optimality Guarantees (2023)0.00
• Adaptive Reward Design For Reinforcement Learning (2024)0.00
• Learning Probabilistic Temporal Logic Specifications For Stochastic Systems (2025)0.00
• Unifying Task Specification In Reinforcement Learning (2016)0.00
• Joint Learning Of Policy With Unknown Temporal Constraints For Safe Reinforcement Learning (2023)0.00