Modular Multi-objective Deep Reinforcement Learning With Decision Values

Abstract

In this work we present a method for using Deep Q-Networks (DQNs) in multi-objective environments. Deep Q-Networks provide remarkable performance in single objective problems learning from high-level visual state representations. However, in many scenarios (e.g in robotics, games), the agent needs to pursue multiple objectives simultaneously. We propose an architecture in which separate DQNs are used to control the agent's behaviour with respect to particular objectives. In this architecture we introduce decision values to improve the scalarization of multiple DQNs into a single action. Our architecture enables the decomposition of the agent's behaviour into controllable and replaceable sub-behaviours learned by distinct modules. Moreover, it allows to change the priorities of particular objectives post-learning, while preserving the overall performance of the agent. To evaluate our solution we used a game-like simulator in which an agent - provided with high-level visual input - pursu

Stats

• citations26
• S2 citations—
• github stars0
• HF likes0
• heat score10.74
• arxiv keytajmajer2017modular

Related papers

• Residual Q-networks For Value Function Factorizing In Multi-agent Reinforcement Learning (2022)10.21
• Deep Q-network Based Multi-agent Reinforcement Learning With Binary Action Agents (2020)0.00
• Modularity Benefits Reinforcement Learning Agents With Competing Homeostatic Drives (2022)0.00
• Multi-objective Deep Reinforcement Learning For Optimisation In Autonomous Systems (2024)0.00
• Q-value Path Decomposition For Deep Multiagent Reinforcement Learning (2020)0.00
• Resolving Implicit Coordination In Multi-agent Deep Reinforcement Learning With Deep Q-networks & Game Theory (2020)0.00
• Macro-action-based Deep Multi-agent Reinforcement Learning (2020)0.00
• Classifying Options For Deep Reinforcement Learning (2016)0.00