Adaptive Reward Design For Reinforcement Learning

Abstract

There is a surge of interest in using formal languages such as Linear Temporal Logic (LTL) to precisely and succinctly specify complex tasks and derive reward functions for Reinforcement Learning (RL). However, existing methods often assign sparse rewards (e.g., giving a reward of 1 only if a task is completed and 0 otherwise). By providing feedback solely upon task completion, these methods fail to encourage successful subtask completion. This is particularly problematic in environments with inherent uncertainty, where task completion may be unreliable despite progress on intermediate goals. To address this limitation, we propose a suite of reward functions that incentivize an RL agent to complete a task specified by an LTL formula as much as possible, and develop an adaptive reward shaping approach that dynamically updates reward functions during the learning process. Experimental results on a range of benchmark RL environments demonstrate that the proposed approach generally outperf

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykwon2024adaptive

Related papers

• Directed Exploration In Reinforcement Learning From Linear Temporal Logic (2024)0.00
• Reward Design For Reinforcement Learning Agents (2025)0.00
• Provably Feedback-efficient Reinforcement Learning Via Active Reward Learning (2023)0.00
• A Policy Search Method For Temporal Logic Specified Reinforcement Learning Tasks (2017)11.58
• Sample-efficient Reinforcement Learning With Temporal Logic Objectives: Leveraging The Task Specification To Guide Exploration (2024)0.00
• Temporal-logic-based Reward Shaping For Continuing Reinforcement Learning Tasks (2020)9.76
• Average Reward Reinforcement Learning For Omega-regular And Mean-payoff Objectives (2025)0.00
• Guiding Multi-agent Multi-task Reinforcement Learning By A Hierarchical Framework With Logical Reward Shaping (2024)0.00