KIPPO: Koopman-inspired Proximal Policy Optimization

Abstract

Reinforcement Learning (RL) has made significant strides in various domains, and policy gradient methods like Proximal Policy Optimization (PPO) have gained popularity due to their balance in performance, training stability, and computational efficiency. These methods directly optimize policies through gradient-based updates. However, developing effective control policies for environments with complex and non-linear dynamics remains a challenge. High variance in gradient estimates and non-convex optimization landscapes often lead to unstable learning trajectories. Koopman Operator Theory has emerged as a powerful framework for studying non-linear systems through an infinite-dimensional linear operator that acts on a higher-dimensional space of measurement functions. In contrast with their non-linear counterparts, linear systems are simpler, more predictable, and easier to analyze. In this paper, we present Koopman-Inspired Proximal Policy Optimization (KIPPO), which learns an approxima

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keycozma2025kippo

Related papers

• Proximal Policy Optimization Algorithms (2017)0.00
• Truly Proximal Policy Optimization (2019)0.00
• Cim-ppo:proximal Policy Optimization With Liu-correntropy Induced Metric (2021)0.00
• Gradient Informed Proximal Policy Optimization (2023)5.15
• Proximal Policy Optimization Via Enhanced Exploration Efficiency (2020)13.70
• A Theoretical Analysis Of Optimistic Proximal Policy Optimization In Linear Markov Decision Processes (2023)0.00
• Policy Optimization With Model-based Explorations (2018)5.84
• Simple Policy Optimization (2024)0.00