A Multimodal Sensing CMOS Imager Based on Dual-Focus Imaging

Advanced machine intelligence is empowered not only by the ever-increasing computational capability for information processing but also by sensors for collecting multimodal information from complex environments. However, simply assembling different sensors can result in bulky systems and complex data processing. Herein, it is shown that a complementary metal-oxide-semiconductor (CMOS) imager can be transformed into a compact multimodal sensing platform through dual-focus imaging. By combining lens-based and lensless imaging, visual information, chemicals, temperature, and humidity can be detected with the same chip and output as a single image. As a proof of concept, the sensor is equipped on a micro-vehicle, and multimodal environmental sensing and mapping is demonstrated. A multimodal endoscope is also developed, and simultaneous imaging and chemical profiling along a porcine digestive tract is achieved. The multimodal CMOS imager is compact, versatile, and extensible and can be widely applied in microrobots, in vivo medical apparatuses, and other microdevices.

Automated summary

By leveraging dual-focus imaging, a complementary metal-oxide-semiconductor (CMOS) imager is transformed into a compact multimodal sensing platform capable of detecting visual information, chemicals, temperature, and humidity, and outputting them as a single image. It has been applied to micro-vehicles for multimodal environmental sensing and mapping, as well as to a multimodal endoscope for simultaneous imaging and chemical profiling along a porcine digestive tract. This multimodal CMOS imager is compact, versatile, and extensible, making it suitable for applications in microrobots, in vivo medical apparatuses, and other microdevices.