The Impact Of Task Underspecification In Evaluating Deep Reinforcement Learning

Abstract

Evaluations of Deep Reinforcement Learning (DRL) methods are an integral part of scientific progress of the field. Beyond designing DRL methods for general intelligence, designing task-specific methods is becoming increasingly prominent for real-world applications. In these settings, the standard evaluation practice involves using a few instances of Markov Decision Processes (MDPs) to represent the task. However, many tasks induce a large family of MDPs owing to variations in the underlying environment, particularly in real-world contexts. For example, in traffic signal control, variations may stem from intersection geometries and traffic flow levels. The select MDP instances may thus inadvertently cause overfitting, lacking the statistical power to draw conclusions about the method's true performance across the family. In this article, we augment DRL evaluations to consider parameterized families of MDPs. We show that in comparison to evaluating DRL methods on select MDP instances, ev

Stats

• citations3
• S2 citations—
• github stars0
• HF likes0
• heat score4.52
• arxiv keyjayawardana2022the

Related papers

• On The Mistaken Assumption Of Interchangeable Deep Reinforcement Learning Implementations (2025)0.00
• Evaluating The Progress Of Deep Reinforcement Learning In The Real World: Aligning Domain-agnostic And Domain-specific Research (2021)0.00
• Unifying Task Specification In Reinforcement Learning (2016)0.00
• Deep Reinforcement Learning At The Edge Of The Statistical Precipice (2021)0.00
• Assessing Generalization In Deep Reinforcement Learning (2018)0.00
• Mitigating Estimation Errors By Twin Td-regularized Actor And Critic For Deep Reinforcement Learning (2023)0.00
• Measuring And Characterizing Generalization In Deep Reinforcement Learning (2018)9.76
• Stealing That Free Lunch: Exposing The Limits Of Dyna-style Reinforcement Learning (2024)0.00