OIL-AD: An Anomaly Detection Framework For Sequential Decision Sequences

Abstract

Anomaly detection in decision-making sequences is a challenging problem due to the complexity of normality representation learning and the sequential nature of the task. Most existing methods based on Reinforcement Learning (RL) are difficult to implement in the real world due to unrealistic assumptions, such as having access to environment dynamics, reward signals, and online interactions with the environment. To address these limitations, we propose an unsupervised method named Offline Imitation Learning based Anomaly Detection (OIL-AD), which detects anomalies in decision-making sequences using two extracted behaviour features: action optimality and sequential association. Our offline learning model is an adaptation of behavioural cloning with a transformer policy network, where we modify the training process to learn a Q function and a state value function from normal trajectories. We propose that the Q function and the state value function can provide sufficient information about

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keywang2024oil

Related papers

• Sequential Anomaly Detection Using Inverse Reinforcement Learning (2020)0.00
• Representation Matters: Offline Pretraining For Sequential Decision Making (2021)0.00
• Towards Data-driven Offline Simulations For Online Reinforcement Learning (2022)0.00
• Rethinking Out-of-distribution Detection For Reinforcement Learning: Advancing Methods For Evaluation And Detection (2024)2.26
• Anomaly Detection Via Learning-based Sequential Controlled Sensing (2023)5.84
• A Scalable Algorithm For Anomaly Detection Via Learning-based Controlled Sensing (2021)0.00
• Anomaly Detection Under Controlled Sensing Using Actor-critic Reinforcement Learning (2020)0.00
• Curriculum Offline Imitation Learning (2021)0.00