Exploring The Impact Of Tunable Agents In Sequential Social Dilemmas

Abstract

When developing reinforcement learning agents, the standard approach is to train an agent to converge to a fixed policy that is as close to optimal as possible for a single fixed reward function. If different agent behaviour is required in the future, an agent trained in this way must normally be either fully or partially retrained, wasting valuable time and resources. In this study, we leverage multi-objective reinforcement learning to create tunable agents, i.e. agents that can adopt a range of different behaviours according to the designer's preferences, without the need for retraining. We apply this technique to sequential social dilemmas, settings where there is inherent tension between individual and collective rationality. Learning a single fixed policy in such settings leaves one at a significant disadvantage if the opponents' strategies change after learning is complete. In our work, we demonstrate empirically that the tunable agents framework allows easy adaption between coop

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyocallaghan2021exploring

Related papers

• Understanding The World To Solve Social Dilemmas Using Multi-agent Reinforcement Learning (2023)0.00
• Prosocial Learning Agents Solve Generalized Stag Hunts Better Than Selfish Ones (2017)0.00
• Learning Through Probing: A Decentralized Reinforcement Learning Architecture For Social Dilemmas (2018)0.00
• Improved Cooperation By Balancing Exploration And Exploitation In Intertemporal Social Dilemma Tasks (2021)0.00
• Evolutionary Multi-agent Reinforcement Learning In Group Social Dilemmas (2024)0.00
• Adapting Behaviour For Learning Progress (2019)0.00
• DSDF: An Approach To Handle Stochastic Agents In Collaborative Multi-agent Reinforcement Learning (2021)0.00
• Towards Cooperation In Sequential Prisoner's Dilemmas: A Deep Multiagent Reinforcement Learning Approach (2018)0.00