 |
High-frequency shape recovery from shading by CNN and domain adaptation |
Importance of structured-light based one-shot scanning technique is increasing because of its simple system configuration and ability of capturing moving objects. One severe limitation of the technique is that it can capture only sparse shape, but not high frequency shapes, because certain area of projection pattern is required to encode spatial information. When densification is performed by interpolation, high-frequency shapes on the object surface are lost, and smooth shapes are recovered. In this paper, we propose a method to recover high-frequency shapes from images using deep learning, where the shapes are measured by a 3D reconstruction method using structured light.
Since RGB images contain the shading information of the object surface, the surface shape can be estimated from the shading. Shape from Shading method requires multiple images with different illumination positions, but in this study, we propose a learning-based approach to recover the shape from a single image. Since estimating shape from shading requires both global and local information, as well as high-resolution output, we adopted an autoencoder with skip connections as the structure of the deep network. In addition, the proportion of high-frequency shape data is small in the publicly available 3D shape dataset, which makes it inefficient for learning such shapes. Therefore, we built a model that creates various high-frequency shapes by superimposing sinusoidal waves, and created a synthetic data set suitable for learning the desired shapes. We propose a new learning method for high-frequency shapes using a sufficient amount of synthetic data and domain adaptation. Quantitative experimental results are presented to confirm the effectiveness of the proposed method. 
Experimental results on shoe soles. There is a high-frequency shape on the surface as shown in Ground Truth (green), but the result of the conventional method does not recover the high-frequency shape as shown in Low-res (light blue), and the shape is smooth. In contrast, our results (yellow) show that the high-frequency shape is recovered. 
Resources
Data: DropBox(High-frequency shape dataset and pre-trained model) Publications
|
| Computer Vision and Graphics Laboratory |