论文标题
用于在线操作检测的基于圆形窗口的级联变压器
A Circular Window-based Cascade Transformer for Online Action Detection
论文作者
论文摘要
在线行动检测旨在基于长期的历史观测值对当前框架进行准确的行动预测。同时，它需要对在线流视频进行实时推断。在本文中，我们主张一个新颖有效的在线行动检测原则。它仅在一个窗口中更新最新，最古老的历史表示，但重复了已经计算的中间表示。基于这一原则，我们引入了一个基于窗口的级联变压器，上面有一个圆形的历史队列，它在每个窗口上都进行了多阶段的关注和级联改善。我们还探讨了在线行动检测与其同行脱机行动分割之间的关联，作为一项辅助任务。我们发现，这种额外的监督有助于歧视历史的聚类，并充当增强功能，以更好地培训分类器和级联改善。我们提出的方法可以在三个具有挑战性的数据集Thumos'14，TVSeries和HDD上实现最先进的性能。接受后，代码将在接受后可用。
Online action detection aims at the accurate action prediction of the current frame based on long historical observations. Meanwhile, it demands real-time inference on online streaming videos. In this paper, we advocate a novel and efficient principle for online action detection. It merely updates the latest and oldest historical representations in one window but reuses the intermediate ones, which have been already computed. Based on this principle, we introduce a window-based cascade Transformer with a circular historical queue, where it conducts multi-stage attentions and cascade refinement on each window. We also explore the association between online action detection and its counterpart offline action segmentation as an auxiliary task. We find that such an extra supervision helps discriminative history clustering and acts as feature augmentation for better training the classifier and cascade refinement. Our proposed method achieves the state-of-the-art performances on three challenging datasets THUMOS'14, TVSeries, and HDD. Codes will be available after acceptance.
