A Study On FGSM Adversarial Training For Neural Retrieval

Abstract

Neural retrieval models have acquired significant effectiveness gains over the last few years compared to term-based methods. Nevertheless, those models may be brittle when faced to typos, distribution shifts or vulnerable to malicious attacks. For instance, several recent papers demonstrated that such variations severely impacted models performances, and then tried to train more resilient models. Usual approaches include synonyms replacements or typos injections -- as data-augmentation -- and the use of more robust tokenizers (characterBERT, BPE-dropout). To further complement the literature, we investigate in this paper adversarial training as another possible solution to this robustness issue. Our comparison includes the two main families of BERT-based neural retrievers, i.e. dense and sparse, with and without distillation techniques. We then demonstrate that one of the most simple adversarial training techniques -- the Fast Gradient Sign Method (FGSM) -- can improve first stage ran

Stats

• citations10
• S2 citations—
• github stars0
• HF likes0
• heat score7.81
• arxiv keylupart2023a

Related papers

• Adaptive Fine-grained Sketch-based Image Retrieval (2022)9.76
• Noisy Self-training With Synthetic Queries For Dense Retrieval (2023)0.00
• Adversarial Sampling And Training For Semi-supervised Information Retrieval (2018)14.43
• Adversarial Reconstruction Feedback For Robust Fine-grained Generalization (2025)0.00
• Black-box Adversarial Attacks Against Dense Retrieval Models: A Multi-view Contrastive Learning Method (2023)9.92
• Federated Learning With Ad-hoc Adapter Insertions: The Case Of Soft-embeddings For Training Classifier-as-retriever (2025)0.00
• RAG-GFM: Overcoming In-memory Bottlenecks In Graph Foundation Models Via Retrieval-augmented Generation (2026)0.00
• Semantic-aware Adversarial Training For Reliable Deep Hashing Retrieval (2023)13.49