论文标题
Pose2RGBD。从绝对位置生成深度和RGB图像
Pose2RGBD. Generating Depth and RGB images from absolute positions
论文作者
论文摘要
我们在计算机视觉和计算机图形字段的交集中提出了一种方法，该方法会根据先前看到的和同步视频，深度和姿势信号自动生成RGBD图像。由于模型必须能够重建纹理（RGB）和结构（深度），因此它会形成场景的隐式表示，而不是明确的表示，例如网格或点云。可以将该过程视为神经渲染，在该过程中，我们获得了一个函数f：pose-> rgbd，我们可以用来浏览生成的场景，与图形模拟类似。我们介绍了两个新数据集，一个基于带有完整地面真相信息的综合数据，而另一个仅使用视频和GPS信号从无人机飞行中记录下来。最后，我们提出了一种仅从视频中生成数据集的完全无监督的方法，以训练Pose2RGBD网络。代码和数据集可在:: https：//gitlab.com/mihaicristianpirvu/pose2rgbd上找到。
We propose a method at the intersection of Computer Vision and Computer Graphics fields, which automatically generates RGBD images using neural networks, based on previously seen and synchronized video, depth and pose signals. Since the models must be able to reconstruct both texture (RGB) and structure (Depth), it creates an implicit representation of the scene, as opposed to explicit ones, such as meshes or point clouds. The process can be thought of as neural rendering, where we obtain a function f : Pose -> RGBD, which we can use to navigate through the generated scene, similarly to graphics simulations. We introduce two new datasets, one based on synthetic data with full ground truth information, while the other one being recorded from a drone flight in an university campus, using only video and GPS signals. Finally, we propose a fully unsupervised method of generating datasets from videos alone, in order to train the Pose2RGBD networks. Code and datasets are available at:: https://gitlab.com/mihaicristianpirvu/pose2rgbd.
