Pirlnav: Pretraining With Imitation And RL Finetuning For Objectnav

Abstract

We study ObjectGoal Navigation -- where a virtual robot situated in a new environment is asked to navigate to an object. Prior work has shown that imitation learning (IL) using behavior cloning (BC) on a dataset of human demonstrations achieves promising results. However, this has limitations -- 1) BC policies generalize poorly to new states, since the training mimics actions not their consequences, and 2) collecting demonstrations is expensive. On the other hand, reinforcement learning (RL) is trivially scalable, but requires careful reward engineering to achieve desirable behavior. We present PIRLNav, a two-stage learning scheme for BC pretraining on human demonstrations followed by RL-finetuning. This leads to a policy that achieves a success rate of $65.0$ on ObjectNav ($+5.0$ absolute over previous state-of-the-art). Using this BC$\to$RL training recipe, we present a rigorous empirical analysis of design choices. First, we investigate whether human demonstrations c