Provably Feedback-efficient Reinforcement Learning Via Active Reward Learning

Abstract

An appropriate reward function is of paramount importance in specifying a task in reinforcement learning (RL). Yet, it is known to be extremely challenging in practice to design a correct reward function for even simple tasks. Human-in-the-loop (HiL) RL allows humans to communicate complex goals to the RL agent by providing various types of feedback. However, despite achieving great empirical successes, HiL RL usually requires too much feedback from a human teacher and also suffers from insufficient theoretical understanding. In this paper, we focus on addressing this issue from a theoretical perspective, aiming to provide provably feedback-efficient algorithmic frameworks that take human-in-the-loop to specify rewards of given tasks. We provide an active-learning-based RL algorithm that first explores the environment without specifying a reward function and then asks a human teacher for only a few queries about the rewards of a task at some state-action pairs. After that, the algorith

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keykong2023provably

Related papers

• Adaptive Reward Design For Reinforcement Learning (2024)0.00
• Reward Design For Reinforcement Learning Agents (2025)0.00
• Active Teacher Selection For Reinforcement Learning From Human Feedback (2023)0.00
• Aligning Humans And Robots Via Reinforcement Learning From Implicit Human Feedback (2025)2.26
• Mapping Out The Space Of Human Feedback For Reinforcement Learning: A Conceptual Framework (2024)0.00
• Information Directed Reward Learning For Reinforcement Learning (2021)0.00
• A Survey Of Reinforcement Learning From Human Feedback (2023)0.00
• Which Rewards Matter? Reward Selection For Reinforcement Learning Under Limited Feedback (2025)0.00