Llm-explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven By Large Language Models

Abstract

Policy exploration is critical in reinforcement learning (RL), where existing approaches include greedy, Gaussian process, etc. However, these approaches utilize preset stochastic processes and are indiscriminately applied in all kinds of RL tasks without considering task-specific features that influence policy exploration. Moreover, during RL training, the evolution of such stochastic processes is rigid, which typically only incorporates a decay in the variance, failing to adjust flexibly according to the agent's real-time learning status. Inspired by the analyzing and reasoning capability of large language models (LLMs), we design LLM-Explorer to adaptively generate task-specific exploration strategies with LLMs, enhancing the policy exploration in RL. In our design, we sample the learning trajectory of the agent during the RL training in a given task and prompt the LLM to analyze the agent's current policy learning status and then generate a probability distribution for future polic

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyhao2025llm

Related papers

• Off-policy Reinforcement Learning With Model-based Exploration Augmentation (2025)0.00
• Agent-pro: Learning To Evolve Via Policy-level Reflection And Optimization (2024)9.59
• Inpainting-guided Policy Optimization For Diffusion Large Language Models (2025)0.00
• From Laws To Motivation: Guiding Exploration Through Law-based Reasoning And Rewards (2024)0.00
• Guiding Reinforcement Learning Using Uncertainty-aware Large Language Models (2024)0.00
• PC-MLP: Model-based Reinforcement Learning With Policy Cover Guided Exploration (2021)0.00
• End-to-end Optimization Of Llm-driven Multi-agent Search Systems Via Heterogeneous-group-based Reinforcement Learning (2025)0.00
• Think Outside The Policy: In-context Steered Policy Optimization (2025)0.00