论文标题
活动表以进行有效的经验重播
Event Tables for Efficient Experience Replay
论文作者
论文摘要
体验重播（ER）是许多深钢筋学习（RL）系统的关键组成部分。但是，来自ER缓冲液的均匀采样会导致收敛缓慢和不稳定的渐近行为。本文介绍了事件表（SSET）的分层采样，该采样将ER缓冲区划分为事件表，每个捕获了最佳行为的重要子序列。我们证明了比传统的整体缓冲方法具有理论上的优势，并将家伙与现有的优先采样策略相结合，以进一步提高学习速度和稳定性。经验结果挑战了Minigrid域，基准RL环境以及高保真赛车模拟器，证明了SSET比现有的ER缓冲液采样方法的优点和多功能性。
Experience replay (ER) is a crucial component of many deep reinforcement learning (RL) systems. However, uniform sampling from an ER buffer can lead to slow convergence and unstable asymptotic behaviors. This paper introduces Stratified Sampling from Event Tables (SSET), which partitions an ER buffer into Event Tables, each capturing important subsequences of optimal behavior. We prove a theoretical advantage over the traditional monolithic buffer approach and combine SSET with an existing prioritized sampling strategy to further improve learning speed and stability. Empirical results in challenging MiniGrid domains, benchmark RL environments, and a high-fidelity car racing simulator demonstrate the advantages and versatility of SSET over existing ER buffer sampling approaches.
