Compact light field photography towards versatile three-dimensional vision

Inspired by natural living systems, modern cameras can attain three-dimensional vision via multi-view geometry like compound eyes in flies, or time-of-flight sensing like echolocation in bats. However, high-speed, accurate three-dimensional sensing capable of scaling over an extensive distance range and coping well with severe occlusions remains challenging. Here, we report compact light field photography for acquiring large-scale light fields with simple optics and a small number of sensors in arbitrary formats ranging from two-dimensional area to single-point detectors, culminating in a dense multi-view measurement with orders of magnitude lower dataload. We demonstrated compact light field photography for efficient multi-view acquisition of time-of-flight signals to enable snapshot three-dimensional imaging with an extended depth range and through severe scene occlusions. Moreover, we show how compact light field photography can exploit curved and disconnected surfaces for real-time non-line-of-sight 3D vision. Compact light field photography will broadly benefit high-speed 3D imaging and open up new avenues in various disciplines. Light field imaging typically requires large format detectors and yet often compromises resolution or speed. Here, compact light field photography is presented to lift both restrictions to see through and around severe occlusions in 3D and real time.

Automated summary

This paper presents a compact light field photography technique that uses simple optics and a small number of sensors to capture large-scale light fields. It enables high-speed and accurate 3D imaging and can handle severe occlusions.