Beyond Freshness And Semantics: A Coupon-collector Framework For Effective Status Updates

Abstract

For status update systems operating over unreliable energy-constrained wireless channels, we address Weaver's long-standing Level-C question: do my packets actually improve the plant's behavior? Each fresh sample carries a stochastic expiration time -- governed by the plant's instability dynamics -- after which the information becomes useless for control. Casting the problem as a coupon-collector variant with expiring coupons, we (i) formulate a two-dimensional average-reward MDP, (ii) prove that the optimal schedule is doubly thresholded in the receiver's freshness timer and the sender's stored lifetime, (iii) derive a closed-form policy for deterministic lifetimes, and (iv) design a Structure-Aware Q-learning algorithm (SAQ) that learns the optimal policy without knowing the channel success probability or lifetime distribution. Simulations validate our theoretical predictions: SAQ matches optimal Value Iteration performance while converging significantly faster than baseline Q-learni

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyahmed2026beyond

Related papers

• Age Of Semantics In Cooperative Communications: To Expedite Simulation Towards Real Via Offline Reinforcement Learning (2022)0.00
• Diffusion Model-based Reinforcement Learning For Version Age Of Information Scheduling: Average And Tail-risk-sensitive Control (2026)0.00
• Optimizing The Long-term Average Reward For Continuing Mdps: A Technical Report (2021)0.00
• A Survey Of Freshness-aware Wireless Networking With Reinforcement Learning (2025)0.00
• Sequential Information Design: Markov Persuasion Process And Its Efficient Reinforcement Learning (2022)0.00
• Revisiting State Augmentation Methods For Reinforcement Learning With Stochastic Delays (2021)10.35
• Learning When To Act: Interval-aware Reinforcement Learning With Predictive Temporal Structure (2026)0.00
• Implications Of Decentralized Q-learning Resource Allocation In Wireless Networks (2017)0.00