Learning To Win, Lose And Cooperate Through Reward Signal Evolution

Abstract

Solving a reinforcement learning problem typically involves correctly prespecifying the reward signal from which the algorithm learns. Here, we approach the problem of reward signal design by using an evolutionary approach to perform a search on the space of all possible reward signals. We introduce a general framework for optimizing \(N\) goals given \(n\) reward signals. Through experiments we demonstrate that such an approach allows agents to learn high-level goals - such as winning, losing and cooperating - from scratch without prespecified reward signals in the game of Pong. Some of the solutions found by the algorithm are surprising, in the sense that they would probably not have been chosen by a person trying to hand-code a given behaviour through a specific reward signal. Furthermore, it seems that the proposed approach may also benefit from higher stability of the training performance when compared with the typical score-based reward signals.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keymuszynski2021learning

Related papers

• Reward Design For Reinforcement Learning Agents (2025)0.00
• Effects Of Different Optimization Formulations In Evolutionary Reinforcement Learning On Diverse Behavior Generation (2021)2.26
• Tiered Reward: Designing Rewards For Specification And Fast Learning Of Desired Behavior (2022)0.00
• An Agent Design With Goal Reaching Guarantees For Enhancement Of Learning (2024)0.00
• Truthful Self-play (2021)0.00
• Reward Design In Cooperative Multi-agent Reinforcement Learning For Packet Routing (2020)0.00
• Scalable Agent Alignment Via Reward Modeling: A Research Direction (2018)0.00
• Reward Learning From Human Preferences And Demonstrations In Atari (2018)0.00