AdaMemento: Adaptive Memory-Assisted Policy Optimization for Reinforcement Learning
Reinforcement Learning
I have read and agree with the venue's withdrawal policy on behalf of myself and my co-authors.
@misc{
sun2024adamemento,
title={AdaMemento: Adaptive Memory-Assisted Policy Optimization for Reinforcement Learning},
author={Shikun Sun},
year={2024},
url={https://openreview.net/forum?id=bWqlWU0kiU}
}
In sparse reward scenarios of reinforcement learning (RL), the memory mechanism provides promising shortcuts to policy optimization by reflecting on past experiences like humans. However, current memory-based RL methods simply store and reuse high-value policies, lacking a deeper refining and filtering of diverse past experiences and hence limiting the capability of memory. In this paper, we propose AdaMemento, an adaptive memory-enhanced RL framework.
Instead of just memorizing positive past experiences, we design a memory-reflection module that exploits both positive and negative experiences by learning to predict known local optimal policies based on real-time states. To effectively gather informative trajectories for the memory, we further introduce a fine-grained intrinsic motivation paradigm, where nuances in similar states can be precisely distinguished to guide exploration. The exploitation of past experiences and exploration of new strategies are then adaptively coordinated by ensemble learning to approach the global optimum. Furthermore, we theoretically prove the superiority of our new intrinsic motivation and ensemble mechanism. From 62 quantitative and visualization experiments, we confirm that AdaMemento can distinguish subtle states for better exploration and effectively exploiting past experiences in memory, achieving significant improvement over previous methods.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
~Shikun_Sun1
Shikun Sun
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Reinforcement Learning
I certify that there is no acknowledgement section in this submission for double blind review.
sun|adamemento_adaptive_memoryassisted_policy_optimization_for_reinforcement_learning
reinforcement learning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
AdaMemento: Adaptive Memory-Assisted Policy Optimization for Reinforcement Learning
ICLR 2025 Conference Withdrawn Submission
ICLR.cc/2025/Conference/Withdrawn_Submission