Multi-intention Inverse Q-learning For Interpretable Behavior Representation

Abstract

In advancing the understanding of natural decision-making processes, inverse reinforcement learning (IRL) methods have proven instrumental in reconstructing animal's intentions underlying complex behaviors. Given the recent development of a continuous-time multi-intention IRL framework, there has been persistent inquiry into inferring discrete time-varying rewards with IRL. To address this challenge, we introduce the class of hierarchical inverse Q-learning (HIQL) algorithms. Through an unsupervised learning process, HIQL divides expert trajectories into multiple intention segments, and solves the IRL problem independently for each. Applying HIQL to simulated experiments and several real animal behavior datasets, our approach outperforms current benchmarks in behavior prediction and produces interpretable reward functions. Our results suggest that the intention transition dynamics underlying complex decision-making behavior is better modeled by a step function instead of a smoothly var

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyzhu2023multi

Related papers

• Inverse Reinforcement Learning With Switching Rewards And History Dependency For Characterizing Animal Behaviors (2025)0.00
• Basis For Intentions: Efficient Inverse Reinforcement Learning Using Past Experience (2022)0.00
• Iq-learn: Inverse Soft-q Learning For Imitation (2021)0.00
• Maximum-likelihood Inverse Reinforcement Learning With Finite-time Guarantees (2022)0.00
• Inverse Reinforcement Learning With Simultaneous Estimation Of Rewards And Dynamics (2016)0.00
• Inverse Reinforcement Learning Via Nonparametric Spatio-temporal Subgoal Modeling (2018)0.00
• Modeling And Interpreting Real-world Human Risk Decision Making With Inverse Reinforcement Learning (2019)0.00
• In-trajectory Inverse Reinforcement Learning: Learn Incrementally Before An Ongoing Trajectory Terminates (2024)5.24