Adaptive Multi-fidelity Reinforcement Learning For Variance Reduction In Engineering Design Optimization

Abstract

Multi-fidelity Reinforcement Learning (RL) frameworks efficiently utilize computational resources by integrating analysis models of varying accuracy and costs. The prevailing methodologies, characterized by transfer learning, human-inspired strategies, control variate techniques, and adaptive sampling, predominantly depend on a structured hierarchy of models. However, this reliance on a model hierarchy can exacerbate variance in policy learning when the underlying models exhibit heterogeneous error distributions across the design space. To address this challenge, this work proposes a novel adaptive multi-fidelity RL framework, in which multiple heterogeneous, non-hierarchical low-fidelity models are dynamically leveraged alongside a high-fidelity model to efficiently learn a high-fidelity policy. Specifically, low-fidelity policies and their experience data are adaptively used for efficient targeted learning, guided by their alignment with the high-fidelity policy. The effectiveness of

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyagrawal2025adaptive

Related papers

• Adaptive Learning Of Design Strategies Over Non-hierarchical Multi-fidelity Models Via Policy Alignment (2024)0.00
• Multi-fidelity Reinforcement Learning Control For Complex Dynamical Systems (2025)0.00
• Vmfer: Von Mises-fisher Experience Resampling Based On Uncertainty Of Gradient Directions For Policy Improvement (2024)0.00
• Is High Variance Unavoidable In RL? A Case Study In Continuous Control (2021)0.00
• Offline Policy Optimization In RL With Variance Regularizaton (2022)0.00
• Provably Efficient Iterated Cvar Reinforcement Learning With Function Approximation And Human Feedback (2023)0.00
• How To Fine-tune The Model: Unified Model Shift And Model Bias Policy Optimization (2023)0.00
• Near-optimal Offline Reinforcement Learning Via Double Variance Reduction (2021)0.00