Switch Trajectory Transformer With Distributional Value Approximation For Multi-task Reinforcement Learning
2022 · Qinjie Lin, Han Liu, Biswa Sengupta
Abstract
We propose SwitchTT, a multi-task extension to Trajectory Transformer but enhanced with two striking features: (i) exploiting a sparsely activated model to reduce computation cost in multi-task offline model learning and (ii) adopting a distributional trajectory value estimator that improves policy performance, especially in sparse reward settings. These two enhancements make SwitchTT suitable for solving multi-task offline reinforcement learning problems, where model capacity is critical for absorbing the vast quantities of knowledge available in the multi-task dataset. More specifically, SwitchTT exploits switch transformer model architecture for multi-task policy learning, allowing us to improve model capacity without proportional computation cost. Also, SwitchTT approximates the distribution rather than the expectation of trajectory value, mitigating the effects of the Monte-Carlo Value estimator suffering from poor sample complexity, especially in the sparse-reward setting. We eva
Authors
(none)
Tags
Stats
Related papers
- Harmodt: Harmony Multi-task Decision Transformer For Offline Reinforcement Learning (2024)0.00
- Enhancing Decision Transformer With Diffusion-based Trajectory Branch Generation (2024)0.00
- Waypoint Transformer: Reinforcement Learning Via Supervised Learning With Intermediate Targets (2023)0.00
- Q-learning Decision Transformer: Leveraging Dynamic Programming For Conditional Sequence Modelling In Offline RL (2022)0.00
- Offline Trajectory Optimization For Offline Reinforcement Learning (2024)1.20
- Updet: Universal Multi-agent Reinforcement Learning Via Policy Decoupling With Transformers (2021)0.00
- Generalized Decision Transformer For Offline Hindsight Information Matching (2021)0.00
- DODT: Enhanced Online Decision Transformer Learning Through Dreamer's Actor-critic Trajectory Forecasting (2024)0.00