Ask-ac: An Initiative Advisor-in-the-loop Actor-critic Framework

Abstract

Despite the promising results achieved, state-of-the-art interactive reinforcement learning schemes rely on passively receiving supervision signals from advisor experts, in the form of either continuous monitoring or pre-defined rules, which inevitably result in a cumbersome and expensive learning process. In this paper, we introduce a novel initiative advisor-in-the-loop actor-critic framework, termed as Ask-AC, that replaces the unilateral advisor-guidance mechanism with a bidirectional learner-initiative one, and thereby enables a customized and efficacious message exchange between learner and advisor. At the heart of Ask-AC are two complementary components, namely action requester and adaptive state selector, that can be readily incorporated into various discrete actor-critic architectures. The former component allows the agent to initiatively seek advisor intervention in the presence of uncertain states, while the latter identifies the unstable states potentially missed by the for

Stats

• citations2
• S2 citations—
• github stars0
• HF likes0
• heat score3.58
• arxiv keyliu2022ask

Related papers

• Adviser-actor-critic: Eliminating Steady-state Error In Reinforcement Learning Control (2025)0.00
• Boosting Exploration In Actor-critic Algorithms By Incentivizing Plausible Novel States (2022)5.24
• Explainable Action Advising For Multi-agent Reinforcement Learning (2022)6.77
• Agent-aware Training For Agent-agnostic Action Advising In Deep Reinforcement Learning (2023)2.26
• Multi-agent Advisor Q-learning (2021)0.00
• Accnet: Actor-coordinator-critic Net For "learning-to-communicate" With Deep Multi-agent Reinforcement Learning (2017)0.00
• Differential Advising In Multi-agent Reinforcement Learning (2020)0.00
• Actor-attention-critic For Multi-agent Reinforcement Learning (2018)0.00