Learning Without Time-based Embodiment Resets In Soft-actor Critic

Abstract

When creating new reinforcement learning tasks, practitioners often accelerate the learning process by incorporating into the task several accessory components, such as breaking the environment interaction into independent episodes and frequently resetting the environment. Although they can enable the learning of complex intelligent behaviors, such task accessories can result in unnatural task setups and hinder long-term performance in the real world. In this work, we explore the challenges of learning without episode terminations and robot embodiment resets using the Soft Actor-Critic (SAC) algorithm. To learn without terminations, we present a continuing version of the SAC algorithm and show that, with simple modifications to the reward functions of existing tasks, continuing SAC can perform as well as or better than episodic SAC while reducing the sensitivity of performance to the value of the discount rate \(\gamma\). On a modified Gym Reacher task, we investigate possible explanat

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyfarrahi2025learning

Related papers

• Context-based Soft Actor Critic For Environments With Non-stationary Dynamics (2021)0.00
• Boosting Soft Actor-critic: Emphasizing Recent Experience Without Forgetting The Past (2019)0.00
• Soft Actor-critic With Inhibitory Networks For Faster Retraining (2022)0.00
• Learning From Demonstrations With SACR2: Soft Actor-critic With Reward Relabeling (2021)0.00
• Regularized Soft Actor-critic For Behavior Transfer Learning (2022)3.58
• Revisiting Discrete Soft Actor-critic (2022)0.00
• Leave No Trace: Learning To Reset For Safe And Autonomous Reinforcement Learning (2017)0.00
• Discrete And Continuous Action Representation For Practical RL In Video Games (2019)0.00