Optimality-based Analysis Of XCSF Compaction In Discrete Reinforcement Learning

Abstract

Learning classifier systems (LCSs) are population-based predictive systems that were originally envisioned as agents to act in reinforcement learning (RL) environments. These systems can suffer from population bloat and so are amenable to compaction techniques that try to strike a balance between population size and performance. A well-studied LCS architecture is XCSF, which in the RL setting acts as a Q-function approximator. We apply XCSF to a deterministic and stochastic variant of the FrozenLake8x8 environment from OpenAI Gym, with its performance compared in terms of function approximation error and policy accuracy to the optimal Q-functions and policies produced by solving the environments via dynamic programming. We then introduce a novel compaction algorithm (Greedy Niche Mass Compaction - GNMC) and study its operation on XCSF's trained populations. Results show that given a suitable parametrisation, GNMC preserves or even slightly improves function approximation error while yi

Stats

• citations6
• S2 citations—
• github stars0
• HF likes0
• heat score6.34
• arxiv keybishop2020optimality

Related papers

• Efficient Use Of Heuristics For Accelerating Xcs-based Policy Learning In Markov Games (2020)0.00
• XQC: Well-conditioned Optimization Accelerates Deep Reinforcement Learning (2025)0.00
• Revisiting Discrete Soft Actor-critic (2022)0.00
• Context-based Soft Actor Critic For Environments With Non-stationary Dynamics (2021)0.00
• A Nearly Optimal And Low-switching Algorithm For Reinforcement Learning With General Function Approximation (2023)0.00
• Learning Impartial Policies For Sequential Counterfactual Explanations Using Deep Reinforcement Learning (2023)0.00
• Evolutionary Reinforcement Learning Via Cooperative Coevolutionary Negatively Correlated Search (2020)9.92
• Improving Exploration In Soft-actor-critic With Normalizing Flows Policies (2019)0.00