Texture Inpainting for Photogrammetric Models

We devise a technique designed to remove the texturing artefacts that are typical of 3D models representing real-world objects, acquired by photogrammetric techniques. Our technique leverages the recent advancements in inpainting of natural colour images, adapting them to the specific context. A neural network, modified and trained for our purposes, replaces the texture areas containing the defects, substituting them with new plausible patches of texels, reconstructed from the surrounding surface texture. We train and apply the network model on locally reparametrized texture patches, so to provide an input that simplifies the learning process, because it avoids any texture seams, unused texture areas, background, depth jumps and so on. We automatically extract appropriate training data from real-world datasets. We show two applications of the resulting method: one, as a fully automatic tool, addressing all problems that can be detected by analysing the UV-map of the input model; and another, as an interactive semi-automatic tool, presented to the user as a 3D 'fixing' brush that has the effect of removing artefacts from any zone the users paints on. We demonstrate our method on a variety of real-world inputs and provide a reference usable implementation.

Automated summary

We propose a technique that removes texturing artefacts in 3D models acquired using photogrammetry. Our technique utilizes recent advancements in inpainting of natural colour images and adapts them to the specific context. By employing a modified and trained neural network, we replace the defective texture areas with plausible patches of texels reconstructed from the surrounding surface texture. The method has two applications: a fully automatic tool that addresses all problems detected by analyzing the UV-map of the input model, and an interactive semi-automatic tool presented as a 3D 'fixing' brush that removes artefacts from any user-painted zone.