Predicting Future Actions Of Reinforcement Learning Agents
2024 · Stephen Chung, Scott Niekum, David Krueger
Abstract
As reinforcement learning agents become increasingly deployed in real-world scenarios, predicting future agent actions and events during deployment is important for facilitating better human-agent interaction and preventing catastrophic outcomes. This paper experimentally evaluates and compares the effectiveness of future action and event prediction for three types of RL agents: explicitly planning, implicitly planning, and non-planning. We employ two approaches: the inner state approach, which involves predicting based on the inner computations of the agents (e.g., plans or neuron activations), and a simulation-based approach, which involves unrolling the agent in a learned world model. Our results show that the plans of explicitly planning agents are significantly more informative for prediction than the neuron activations of the other types. Furthermore, using internal plans proves more robust to model quality compared to simulation-based approaches when predicting actions, while th
Authors
(none)
Tags
Stats
Related papers
- Learning Dynamics Model In Reinforcement Learning By Incorporating The Long Term Future (2019)0.00
- An Empirical Study On The Power Of Future Prediction In Partially Observable Environments (2024)0.00
- Ego-foresight: Self-supervised Learning Of Agent-aware Representations For Improved RL (2024)0.00
- Assessing Human Interaction In Virtual Reality With Continually Learning Prediction Agents Based On Reinforcement Learning Algorithms: A Pilot Study (2021)0.00
- Episodic Future Thinking Mechanism For Multi-agent Reinforcement Learning (2024)0.00
- Explaining An Agent's Future Beliefs Through Temporally Decomposing Future Reward Estimators (2024)0.00
- Data-efficient Reinforcement Learning With Self-predictive Representations (2020)0.00
- Plan To Predict: Learning An Uncertainty-foreseeing Model For Model-based Reinforcement Learning (2023)0.00