论文标题
UNIMSE：迈向统一的多模式分析和情感识别
UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition
论文作者
论文摘要
对话（ERC）中的多模式情感分析（MSA）和情绪识别是计算机了解人类行为的关键研究主题。从心理的角度来看，情绪是短时间情感或感受的表达，而情感的形成并保持了更长的时间。但是，大多数现有的作品分别研究情感和情感，并不能完全利用两者背后的互补知识。在本文中，我们提出了一个多模式情感知识共享框架（UNIMSE），该框架将MSA和ERC任务从功能，标签和模型统一。我们在句法和语义层面上执行模态融合，并在样本和样本之间引入对比度学习，以更好地捕捉情感和情感之间的差异和一致性。在四个公共基准数据集（MOSI，MOSEI，MELD和IEMOCAP）上进行的实验证明了该方法的有效性，并且与最先进的方法相比，进行了一致的改进。
Multimodal sentiment analysis (MSA) and emotion recognition in conversation (ERC) are key research topics for computers to understand human behaviors. From a psychological perspective, emotions are the expression of affect or feelings during a short period, while sentiments are formed and held for a longer period. However, most existing works study sentiment and emotion separately and do not fully exploit the complementary knowledge behind the two. In this paper, we propose a multimodal sentiment knowledge-sharing framework (UniMSE) that unifies MSA and ERC tasks from features, labels, and models. We perform modality fusion at the syntactic and semantic levels and introduce contrastive learning between modalities and samples to better capture the difference and consistency between sentiments and emotions. Experiments on four public benchmark datasets, MOSI, MOSEI, MELD, and IEMOCAP, demonstrate the effectiveness of the proposed method and achieve consistent improvements compared with state-of-the-art methods.
