Single-Image Real-Time Rain Removal Based on Depth-Guided Non-Local Features

Rain is a common weather phenomenon that affects environmental monitoring and surveillance systems. According to an established rain model (Garg and Nayar, 2007), the scene visibility in the rain varies with the depth from the camera, where objects faraway are visually blocked more by the fog than by the rain streaks. However, existing datasets and methods for rain removal ignore these physical properties, thus limiting the rain removal efficiency on real photos. In this work, we analyze the visual effects of rain subject to scene depth and formulate a rain imaging model that collectively considers rain streaks and fog. Also, we prepare a dataset called RainCityscapes on real outdoor photos. Furthermore, we design a novel real-time end-to-end deep neural network, for which we train to learn the depth-guided non-local features and to regress a residual map to produce a rain-free output image. We performed various experiments to visually and quantitatively compare our method with several state-of-the-art methods to show its superiority over others.

Automated summary

Analyzing the visual effects of rain based on scene depth, a comprehensive rain imaging model was formulated considering both rain streaks and fog. Through a real-time end-to-end deep neural network, a dataset named RainCityscapes was prepared for real outdoor photos, achieving superior performance compared to other state-of-the-art methods in both visual and quantitative evaluations.