论文标题
社交互动中的身体行为：新颖的注释和最新评估
Bodily Behaviors in Social Interaction: Novel Annotations and State-of-the-Art Evaluation
论文作者
论文摘要
肢体语言是一种引人注目的社交信号，其自动分析可以大大推动人工智能系统，以理解和积极参与社交互动。尽管计算机视觉在诸如头部和身体姿势估计之类的低级任务中取得了令人印象深刻的进步，但探索诸如示意，修饰或摸索之类的更微妙行为的发现尚未得到很好的探索。在本文中，我们介绍了BBSI，这是复杂的身体行为的第一组注释，嵌入了小组环境中的连续社交互动中。根据心理学的先前工作，我们在Mpiigroupaction数据集中手动注释了26个小时的自发人类行为，并具有15种不同的肢体语言类别。我们介绍了有关数据集的全面描述性统计数据以及注释质量评估的结果。为了自动检测这些行为，我们适应了金字塔扩张的注意网络（PDAN），这是一种最新的人类动作检测方法。我们使用四个空间特征的四种变体作为PDAN的输入进行实验：两流膨胀的3D CNN，时间段网络，时间移位模块和Swin Transformer。结果是有希望的，这表明了这项艰巨的任务改进的好空间。 BBSI代表了自动理解社会行为的难题中的关键作品，研究界完全可以使用。
Body language is an eye-catching social signal and its automatic analysis can significantly advance artificial intelligence systems to understand and actively participate in social interactions. While computer vision has made impressive progress in low-level tasks like head and body pose estimation, the detection of more subtle behaviors such as gesturing, grooming, or fumbling is not well explored. In this paper we present BBSI, the first set of annotations of complex Bodily Behaviors embedded in continuous Social Interactions in a group setting. Based on previous work in psychology, we manually annotated 26 hours of spontaneous human behavior in the MPIIGroupInteraction dataset with 15 distinct body language classes. We present comprehensive descriptive statistics on the resulting dataset as well as results of annotation quality evaluations. For automatic detection of these behaviors, we adapt the Pyramid Dilated Attention Network (PDAN), a state-of-the-art approach for human action detection. We perform experiments using four variants of spatial-temporal features as input to PDAN: Two-Stream Inflated 3D CNN, Temporal Segment Networks, Temporal Shift Module and Swin Transformer. Results are promising and indicate a great room for improvement in this difficult task. Representing a key piece in the puzzle towards automatic understanding of social behavior, BBSI is fully available to the research community.
