Abstract
Transformer-based pre-trained language models (PLMs) excel in text classification but suffer from attention dilution and attention sink effects, forcing models to over-focus on task-irrelevant tokens. Existing attention supervision methods rely on costly token-level human annotations or static heuristics, which fail to scale or capture context-dependent token importance. To address this, we propose AttCal, a self-supervised, annotation-free attention calibration framework via task-space sensitivity feedback. AttCal treats attention distributions as stochastic policies to guide token deletion perturbations, infers context-dependent token importance from semantic shifts, and optimizes attention parameters via policy gradient. Critically, AttCal freezes the PLM backbone and updates only attention projection layers to ensure training efficiency. Benchmarks across five datasets demonstrate that AttCal significantly outperforms vanilla PLMs-yielding up to a 15.8% accuracy lift for Llama3-8B-and surpasses SOTA supervised baselines in both classification and interpretability.