A Generalized Algorithm For Multi-objective Reinforcement Learning And Policy Adaptation

Abstract

We introduce a new algorithm for multi-objective reinforcement learning (MORL) with linear preferences, with the goal of enabling few-shot adaptation to new tasks. In MORL, the aim is to learn policies over multiple competing objectives whose relative importance (preferences) is unknown to the agent. While this alleviates dependence on scalar reward design, the expected return of a policy can change significantly with varying preferences, making it challenging to learn a single model to produce optimal policies under different preference conditions. We propose a generalized version of the Bellman equation to learn a single parametric representation for optimal policies over the space of all possible preferences. After an initial learning phase, our agent can execute the optimal policy under any given preference, or automatically infer an underlying preference with very few samples. Experiments across four different domains demonstrate the effectiveness of our approach.

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyyang2019a

Related papers

• Sample-efficient Multi-objective Learning Via Generalized Policy Improvement Prioritization (2023)5.24
• Provable Multi-objective Reinforcement Learning With Generative Models (2020)0.00
• On Generalization Across Environments In Multi-objective Reinforcement Learning (2025)0.00
• Interpretability By Design For Efficient Multi-objective Reinforcement Learning (2025)0.00
• Navigating Trade-offs: Policy Summarization For Multi-objective Reinforcement Learning (2024)2.26
• Toward Negotiable Reinforcement Learning: Shifting Priorities In Pareto Optimal Sequential Decision-making (2017)0.00
• Addressing The Issue Of Stochastic Environments And Local Decision-making In Multi-objective Reinforcement Learning (2022)0.00
• Utility-based Reinforcement Learning: Unifying Single-objective And Multi-objective Reinforcement Learning (2024)2.26