论文标题
Flow2Stereo：有效的自我监督学习光流和立体声匹配的学习
Flow2Stereo: Effective Self-Supervised Learning of Optical Flow and Stereo Matching
论文作者
论文摘要
在本文中，我们提出了一种联合学习光流和立体声匹配的统一方法。我们的第一个直觉是立体声匹配可以建模为光流的特殊情况，我们可以利用立体视频后面的3D几何形状来指导学习这两种形式的对应关系。然后，我们将这些知识注入到最先进的自我监督学习框架中，并训练一个网络以估计流量和立体声。其次，我们在先前的自我监督学习方法中揭示了瓶颈，并建议创建一套新的挑战性代理任务以提高绩效。这两个见解产生了一个单个模型，该模型在Kitti 2012和2015基准的所有现有无监督流和立体声方法中都达到了最高精度。更值得注意的是，我们的自我监管方法甚至胜过几种完全监督的方法，包括PWC-NET和FLOWNET2在Kitti 2012上。
In this paper, we propose a unified method to jointly learn optical flow and stereo matching. Our first intuition is stereo matching can be modeled as a special case of optical flow, and we can leverage 3D geometry behind stereoscopic videos to guide the learning of these two forms of correspondences. We then enroll this knowledge into the state-of-the-art self-supervised learning framework, and train one single network to estimate both flow and stereo. Second, we unveil the bottlenecks in prior self-supervised learning approaches, and propose to create a new set of challenging proxy tasks to boost performance. These two insights yield a single model that achieves the highest accuracy among all existing unsupervised flow and stereo methods on KITTI 2012 and 2015 benchmarks. More remarkably, our self-supervised method even outperforms several state-of-the-art fully supervised methods, including PWC-Net and FlowNet2 on KITTI 2012.
