Beyond Rewards In Reinforcement Learning For Cyber Defence

Abstract

Recent years have seen an explosion of interest in autonomous cyber defence agents trained to defend computer networks using deep reinforcement learning. These agents are typically trained in cyber gym environments using dense, highly engineered reward functions which combine many penalties and incentives for a range of (un)desirable states and costly actions. Dense rewards help alleviate the challenge of exploring complex environments but risk biasing agents towards suboptimal and potentially riskier solutions, a critical issue in complex cyber environments. We thoroughly evaluate the impact of reward function structure on learning and policy behavioural characteristics using a variety of sparse and dense reward functions, two well-established cyber gyms, a range of network sizes, and both policy gradient and value-based RL algorithms. Our evaluation is enabled by a novel ground truth evaluation approach which allows directly comparing between different reward functions, illuminating

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keybates2026beyond

Related papers

• Reward Shaping For Happier Autonomous Cyber Security Agents (2023)9.23
• Deep Reinforcement Learning For Autonomous Cyber Defence: A Survey (2023)0.00
• Learning To Communicate In Multi-agent Reinforcement Learning For Autonomous Cyber Defence (2025)0.00
• A Study On Dense And Sparse (visual) Rewards In Robot Policy Learning (2021)0.00
• Learning To Cope With Adversarial Attacks (2019)0.00
• Disturbing Reinforcement Learning Agents With Corrupted Rewards (2021)0.00
• Cygil: A Cyber Gym For Training Autonomous Agents Over Emulated Network Systems (2021)0.00
• Unveiling The Black Box: A Multi-layer Framework For Explaining Reinforcement Learning-based Cyber Agents (2025)0.00