MAVRL: Learning Reward Functions From Multiple Feedback Types With Amortized Variational Inference

Abstract

Reward learning typically relies on a single feedback type or combines multiple feedback types using manually weighted loss terms. Currently, it remains unclear how to jointly learn reward functions from heterogeneous feedback types such as demonstrations, comparisons, ratings, and stops that provide qualitatively different signals. We address this challenge by formulating reward learning from multiple feedback types as Bayesian inference over a shared latent reward function, where each feedback type contributes information through an explicit likelihood. We introduce a scalable amortized variational inference approach that learns a shared reward encoder and feedback-specific likelihood decoders and is trained by optimizing a single evidence lower bound. Our approach avoids reducing feedback to a common intermediate representation and eliminates the need for manual loss balancing. Across discrete and continuous-control benchmarks, we show that jointly inferred reward posteriors outperf

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybaur2026mavrl

Related papers

• VIREL: A Variational Inference Framework For Reinforcement Learning (2018)0.00
• Invariance In Policy Optimisation And Partial Identifiability In Reward Learning (2022)0.00
• Variational Inference For Model-free And Model-based Reinforcement Learning (2022)0.00
• Generalizing Across Multi-objective Reward Functions In Deep Reinforcement Learning (2018)0.00
• The Implicit Curriculum: Learning Dynamics In RL With Verifiable Rewards (2026)0.00
• Provably Feedback-efficient Reinforcement Learning Via Active Reward Learning (2023)0.00
• Policy Improvement Reinforcement Learning (2026)0.00
• Distributional Reinforcement Learning For Multi-dimensional Reward Functions (2021)0.00