A higher performance shape from focus strategy based on unsupervised deep learning for 3D shape reconstruction

Shape From Focus (SFF) is one of the most popular strategies for reconstructing object's 3D shape, which doesn't require any additional technology. SFF strategies generate the object's 3D shape using a sequence of 2D images with the same field of perspective and various areas in-focus. Although SFF has many literature studies and is one of the most preferred strategies in computer vision to produce 3D shapes of objects, this research field still contains critical shortcomings such as high computational costs and complexity, less effectiveness in noisy or poorly textured areas, insufficient focus information to be extracted from the input images, generally adapting a pre- or post-processing technique and using only gray levels of original images to acquire the pixel's focus values. In order to minimize these shortcomings, an unsupervised deep learning-based SFF strategy that gives higher performance than current strategies is suggested in this study. When compared with the studies generated for 3D shape reconstruction in the literature, the proposed SFF strategy provides various fundamental contributions such as the first study proposing SFF strategy based on unsupervised deep learning, providing a high-quality focus measurement operator, acquiring the pixel's focus values from deep features and not requiring any pre- or post-processing technique. In order to assess the efficiency of the proposed SFF strategy, well-known focus measurement operators are analyzed using 4 different synthetic and microscope image sequences. In order to determine which SFF strategy can extract more essential details from the 2D images with various areas in-focus on these sequences, quality assessment criteria with and without requiring a ground-truth are preferred, which are Root Mean Square Error (RMSE), Peak Signal Noise Ratio (PSNR), Universal Quality Index (UQI), Correlation Coefficient (CC), Standard Deviation (SD) and Kurtosis Metric (KM). Both qualitative and quantitative evaluations reveal that the suggested SFF strategy with highest PSNR (28.1725, 25.0854), UQI (0.9454, 0.7543), CC (0.9457, 0.7132), and lowest RMSE (0.0390, 0.0563), SD (3.1666, 1.1768), KM (2.1254, 3.6362) values produces higher performance, and the designed unsupervised deep learning model is more efficient to transmit crucial details from 2D images.

Automated summary

Shape From Focus (SFF) is a popular strategy for reconstructing object's 3D shape, but it has several shortcomings. This study proposes an unsupervised deep learning-based SFF strategy that offers higher performance and quality.