Scale-invariant Temporal History (SITH): Optimal Slicing Of The Past In An Uncertain World

Abstract

In both the human brain and any general artificial intelligence (AI), a representation of the past is necessary to predict the future. However, perfect storage of all experiences is not feasible. One approach utilized in many applications, including reward prediction in reinforcement learning, is to retain recently active features of experience in a buffer. Despite its prior successes, we show that the fixed length buffer renders Deep Q-learning Networks (DQNs) fragile to changes in the scale over which information can be learned. To enable learning when the relevant temporal scales in the environment are not known *a priori*, recent advances in psychology and neuroscience suggest that the brain maintains a compressed representation of the past. Here we introduce a neurally-plausible, scale-free memory representation we call Scale-Invariant Temporal History (SITH) for use with artificial agents. This representation covers an exponentially large period of time by sacrificing temporal ac

Stats

• citations0
• S2 citations—
• github stars0
• HF likes0
• heat score0.00
• arxiv keyspears2017scale

Related papers

• Deep Reinforcement Learning With Time-scale Invariant Memory (2024)0.00
• Towards Mental Time Travel: A Hierarchical Memory For Reinforcement Learning Agents (2021)0.00
• Estimating Scale-invariant Future In Continuous Time (2018)9.59
• Learning When To Act: Interval-aware Reinforcement Learning With Predictive Temporal Structure (2026)0.00
• Temporal Alignment For History Representation In Reinforcement Learning (2022)0.00
• Episodic Memory For Learning Subjective-timescale Models (2020)0.00
• Dynamic Memory For Interpretable Sequential Optimisation (2022)0.00
• Self-supervised Exploration Via Temporal Inconsistency In Reinforcement Learning (2022)3.58