DSDNet: Toward single image deraining with self-paced curricular dual stimulations

A crucial challenge regarding the single image deraining task is to completely remove rain streaks while still preserving explicit image details. Due to the inherent overlapping between rain streaks and background scenes, the texture details could be inevitably lost when clearing rain away from the degraded image, making the two purposes contradictory. Existing deep learning based approaches endeavor to resolve the two issues successively in a cascaded framework or to treat them as independent tasks in a parallel structure. However, none of the models explores a proper interaction between rain distributions and hidden feature responses, which intuitively would provide more clues to facilitate the procedures of rain streak removal as well as detail restoration. In this paper, we investigate the impact of rain streak detection for single image deraining and propose a novel deep network with dual stimulations, namely, DSDNet. The proposed DSDNet utilizes a dual-stream pipeline to separately estimate rain streaks and a loss of details, and more importantly, an additional mask that indicates both location and intensity of rains is jointly predicted. In particular, the rain mask is involved in a tailored stimulation strategy that is deployed into each stream of the proposed model, serving as guidance for allowing the network to focus on rain removal and detail recovery in rain regions rather than non-rain areas. Moreover, we incorporate a self-paced semi-curriculum learning design to alleviate the learning ambiguity brought by the prediction of the rain mask and thus accelerate the training process. Extensive experiments demonstrate the proposed method outperforms the state-of-the-art methods on several benchmarks, including in both synthetic and real-world scenarios. The effectiveness of the proposed method is also validated via joint single image deraining, detection, and segmentation tasks.

Automated summary

The crucial challenge of single image deraining is to remove rain streaks while preserving image details. This paper proposes a novel deep network called DSDNet, which estimates rain streaks and detail loss separately, and predicts a rain mask indicating the location and intensity of rain. Extensive experiments show that the proposed method outperforms state-of-the-art methods and is effective in joint tasks of single image deraining, detection, and segmentation.