Abstract
We introduce Train/Test-Time Adaptation with Retrieval (\(\{\rm T^3AR\}\)), a method to adapt models both at train and test time by means of a retrieval module and a searchable pool of external samples. Before inference, \(\{\rm T^3AR\}\) adapts a given model to the downstream task using refined pseudo-labels and a self-supervised contrastive objective function whose noise distribution leverages retrieved real samples to improve feature adaptation on the target data manifold. The retrieval of real images is key to \(\{\rm T^3AR\}\) since it does not rely solely on synthetic data augmentations to compensate for the lack of adaptation data, as typically done by other adaptation algorithms. Furthermore, thanks to the retrieval module, our method gives the user or service provider the possibility to improve model adaptation on the downstream task by incorporating further relevant data or to fully remove samples that may no longer be available due to changes in user preference after deploym