论文标题

轻巧的HDR摄像头ISP,用于通过傅立叶对抗网络在动态照明条件下的强大感知

Lightweight HDR Camera ISP for Robust Perception in Dynamic Illumination Conditions via Fourier Adversarial Networks

论文作者

Shyam, Pranjay, Sengar, Sandeep Singh, Yoon, Kuk-Jin, Kim, Kyung-Soo

论文摘要

商业紧凑型摄像机传感器的动态动态范围有限,导致不准确的照明条件的场景表示,对图像质量产生不利影响,并随后限制了基础图像处理算法的性能。当前最新的(SOTA)卷积神经网络(CNN)是作为后处理技术开发的,可以独立恢复未接触的图像。但是,当应用于包含现实世界下降的图像,例如眩光,高光束,颜色出血,噪声强度不同时,这些算法会扩大降解,从而进一步降低图像质量。我们提出了一种轻巧的两阶段图像增强算法,使用频率先验依次平衡照明和降噪,以克服这些限制。此外,为了确保逼真的图像质量,我们利用图像的频率和空间域性质之间的关系,并提出一个基于傅立叶频谱的对抗框架(AFNET),以在不同的照明条件下保持一致的图像增强。虽然当前的图像增强功能被视为后处理技术,但我们检查是否可以扩展这种算法,以将图像信号处理(ISP)管道的功能集成到从原始传感器数据中受益和轻量级CNN体系结构的相机传感器中。基于定量和定性评估,我们还研究了图像增强技术对在不同照明条件下的对象检测和语义分割等共同感知任务执行的实用性和影响。

The limited dynamic range of commercial compact camera sensors results in an inaccurate representation of scenes with varying illumination conditions, adversely affecting image quality and subsequently limiting the performance of underlying image processing algorithms. Current state-of-the-art (SoTA) convolutional neural networks (CNN) are developed as post-processing techniques to independently recover under-/over-exposed images. However, when applied to images containing real-world degradations such as glare, high-beam, color bleeding with varying noise intensity, these algorithms amplify the degradations, further degrading image quality. We propose a lightweight two-stage image enhancement algorithm sequentially balancing illumination and noise removal using frequency priors for structural guidance to overcome these limitations. Furthermore, to ensure realistic image quality, we leverage the relationship between frequency and spatial domain properties of an image and propose a Fourier spectrum-based adversarial framework (AFNet) for consistent image enhancement under varying illumination conditions. While current formulations of image enhancement are envisioned as post-processing techniques, we examine if such an algorithm could be extended to integrate the functionality of the Image Signal Processing (ISP) pipeline within the camera sensor benefiting from RAW sensor data and lightweight CNN architecture. Based on quantitative and qualitative evaluations, we also examine the practicality and effects of image enhancement techniques on the performance of common perception tasks such as object detection and semantic segmentation in varying illumination conditions.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源