Elo-rated Sequence Rewards: Advancing Reinforcement Learning Models

Abstract

Reinforcement Learning (RL) heavily relies on the careful design of the reward function. However, accurately assigning rewards to each state-action pair in Long-Term Reinforcement Learning (LTRL) tasks remains a significant challenge. As a result, RL agents are often trained under expert guidance. Inspired by the ordinal utility theory in economics, we propose a novel reward estimation algorithm: ELO-Rating based Reinforcement Learning (ERRL). This approach features two key contributions. First, it uses expert preferences over trajectories rather than cardinal rewards (utilities) to compute the ELO rating of each trajectory as its reward. Second, a new reward redistribution algorithm is introduced to alleviate training instability in the absence of a fixed anchor reward. In long-term scenarios (up to 5000 steps), where traditional RL algorithms struggle, our method outperforms several state-of-the-art baselines. Additionally, we conduct a comprehensive analysis of how expert preference

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyju2024elo

Related papers

• Reward Design For Reinforcement Learning Agents (2025)0.00
• Adaptive Reward Design For Reinforcement Learning (2024)0.00
• Differentiable Evolutionary Reinforcement Learning (2025)0.00
• Listwise Reward Estimation For Offline Preference-based Reinforcement Learning (2024)0.00
• Reinforcement Learning With Non-ergodic Reward Increments: Robustness Via Ergodicity Transformations (2023)0.00
• Replacing Rewards With Examples: Example-based Policy Search Via Recursive Classification (2021)0.00
• Average Reward Reinforcement Learning For Omega-regular And Mean-payoff Objectives (2025)0.00
• Information Directed Reward Learning For Reinforcement Learning (2021)0.00