Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second

Non-line-of-sight imaging can recover the 3D geometry of hidden objects, but is limited by weak multibounce signals. Here, the authors introduce a multipixel time-of-flight NLOS imaging approach, combining array detectors and a fast algorithm, for live reconstruction of natural nonretroreflective objects. Non-Line-Of-Sight (NLOS) imaging aims at recovering the 3D geometry of objects that are hidden from the direct line of sight. One major challenge with this technique is the weak available multibounce signal limiting scene size, capture speed, and reconstruction quality. To overcome this obstacle, we introduce a multipixel time-of-flight non-line-of-sight imaging method combining specifically designed Single Photon Avalanche Diode (SPAD) array detectors with a fast reconstruction algorithm that captures and reconstructs live low-latency videos of non-line-of-sight scenes with natural non-retroreflective objects. We develop a model of the signal-to-noise-ratio of non-line-of-sight imaging and use it to devise a method that reconstructs the scene such that signal-to-noise-ratio, motion blur, angular resolution, and depth resolution are all independent of scene depth suggesting that reconstruction of very large scenes may be possible.

Automated summary

The study introduces a multipixel time-of-flight non-line-of-sight imaging approach combining array detectors and a fast algorithm, allowing for live reconstruction of scenes with natural non-retroreflective objects.