VGenNet: Variable Generative Prior Enhanced Single Pixel Imaging

Single-pixel imaging (SPI) is an emerging imaging method-ology that converts a two-or even three-dimensional image acquisition problem into a one-dimensional (1D) temporal-signal detection problem. Thus, it is crucially important to develop efficient SPI techniques for image reconstruction from the 1D measurements, in particular, an undersampled one. Recently, various studies have demonstrated the superiority of deep learning for SPI. However, due to the generalization issue, conventional data-driven deep learning is a task-specific approach. One needs to retrain the neural network for different SPI imaging problems and different types of objects. Here, we propose a variable generative network enhanced SPI algorithm (VGenNet) by incorporating a model-driven fine-tuning process into a generative model that may have been trained for other tasks. VGenNet simultaneously updates the input vector and the weights in a generator to generate feasible solutions that reproduce the raw measurements. We demonstrate the proposed technique with indoor SPI and outdoor 3D single-pixel LiDAR experiments. Our results show that high-quality images can be reconstructed at low sampling ratios under different system configurations, demonstrating the good performance and flexibility of VGenNet. Overall, the proposed VGenNet is a general framework to take advantage of both the data and physics priors, allowing the direct use of a pretrained generative model to solve various inverse imaging problems.

Automated summary

Single-pixel imaging (SPI) converts a multi-dimensional image acquisition problem into a one-dimensional temporal-signal detection problem, and efficient SPI techniques are crucial for image reconstruction. Deep learning has shown superiority in SPI, but is task-specific and requires retraining for different problems. The proposed VGenNet algorithm incorporates a model-driven fine-tuning process into a generative model, allowing the use of a pretrained model for various inverse imaging problems. Experimental results demonstrate the high-quality image reconstruction and flexibility of VGenNet.