论文标题
通过对比度学习的混合判别生成培训
Hybrid Discriminative-Generative Training via Contrastive Learning
论文作者
论文摘要
对比学习和监督学习都取得了重大进步和成功。但是，到目前为止，它们在很大程度上被视为两个独立的目标，仅通过共享神经网络而汇总。在本文中，我们表明，通过对基于能量的模型的混合判别生成培训的观点，我们可以在对比度学习和监督学习之间建立直接联系。除了呈现这种统一的观点外，我们还显示了我们对基于能量的损失的近似选择的特定选择，在CIFAR-10和CIFAR-100上的分类准确性方面优于现有实践。它还可以提高鲁棒性，分布外检测和校准的性能。
Contrastive learning and supervised learning have both seen significant progress and success. However, thus far they have largely been treated as two separate objectives, brought together only by having a shared neural network. In this paper we show that through the perspective of hybrid discriminative-generative training of energy-based models we can make a direct connection between contrastive learning and supervised learning. Beyond presenting this unified view, we show our specific choice of approximation of the energy-based loss outperforms the existing practice in terms of classification accuracy of WideResNet on CIFAR-10 and CIFAR-100. It also leads to improved performance on robustness, out-of-distribution detection, and calibration.
