Multimodal light-sensing pixel arrays

Image sensors used in consumer cameras only detect intensity of light. Spectral, angular, and phase information of incident light is largely lost. For conventional applications like photography, the intensity information with three color bands (RGB) is sufficient. However, for advanced sensing applications, such as autonomous vehicles, biomedical imaging, and robotics, extracting more information from the incident light could help machines to make better decisions. It is expected that, in the future, more cameras will be built for machines than for people, which will be further accelerated by the rapid progress in machine learning and artificial intelligence. We envision that these applications will substantially benefit from multimodal measurement of light fields by advanced imaging sensors. In this Perspective, we discuss recent progress in image sensors with multimodal capabilities. Published under an exclusive license by AIP Publishing.

Automated summary

Traditional consumer cameras only detect the intensity of light, while advanced applications like autonomous vehicles, biomedical imaging, and robotics could benefit from extracting more information such as spectral, angular, and phase information from incident light.