DENAO: Monocular Depth Estimation Network With Auxiliary Optical Flow

Estimating depth from multi-view images captured by a localized monocular camera is an essential task in computer vision and robotics. In this study, we demonstrate that learning a convolutional neural network (CNN) for depth estimation with an auxiliary optical flow network and the epipolar geometry constraint can greatly benefit the depth estimation task and in turn yield large improvements in both accuracy and speed. Our architecture is composed of two tightly-coupled encoder-decoder networks, i.e., an optical flow net and a depth net, the core part being a list of exchange blocks between the two nets and an epipolar feature layer in the optical flow net to improve predictions of both depth and optical flow. Our architecture allows to input arbitrary number of multiview images with a linearly growing time cost for optical flow and depth estimation. Experimental result on five public datasets demonstrates that our method, named DENAO, runs at 38.46fps on a single Nvidia TITAN Xp GPU which is 5.15X similar to 142X faster than the state-of-the-art depth estimation methods Meanwhile, our DENAO can concurrently output predictions of both depth and optical flow, and performs on par with or outperforms the state-of-the-art depth estimation methods and optical flow methods.

Automated summary

This study demonstrates that learning a convolutional neural network for depth estimation, along with an auxiliary optical flow network and epipolar geometry constraint, can greatly improve accuracy and speed. The architecture consists of tightly coupled encoder-decoder networks for optical flow and depth, with exchange blocks and an epipolar feature layer to enhance predictions. The method DENAO runs at high speed on a single GPU, outperforming state-of-the-art depth estimation and optical flow methods.