Underwater projected pattern stereo using Multi-scale CNN
Recently, demands on underwater 3D reconstruction are increasing in the field of robot vision, marine biology, sports science, and so on. We propose a technique to capture accurate and dense underwater dynamic scene. Since there are several problems such as attenuation, refraction and underwater disturbances like bubbles, same method as in the air cannon be applied directly. In this research we present a measurement technique suitable for underwater environment, which is robust against such disturbances by using several methods shown below. Furthermore, we captured live moving fish and swimming human body to confirm the feasibility.

Projected pattern stereo

The biggest problem in underwater measurement, is attenuation of visible lights. Since attenuation degrades contrast, active stereo method is robust against attenuation because it can detect projected pattern in low-contrast environment. However in general, dynamic active stereo method cannot reconstruct dense shape and its algorithm is complicated because of refraction effect. Therefore, we used projected pattern stereo, which projects pattern to add artificial texture and contrast, and reconstruct with passive stereo. This method can reconstruct dense shape with simple algorithm because it can ignore refraction of light source.
System configuration of projected pattern stereo

Refraction undistort by approximation

Cameras are usually set into waterproof housing, and refraction occurs its surface. Epipolar line will be horizontal in parallel stereo without distortion, but it will not be if refraction occurs, and it makes passive stereo difficult. A method to absorb refraction distortion by focal length adjustment and radial distortion model is widely known, we investigated depth-range where refraction is effectively absorbed by confirming approximation accuracy in simulation.
Relationship between depth and approximation error

Robust stereo matching by Multi-scale CNN stereo

There are specific disturbances such as bubbles or shadows of water surface in underwater environment, which degrades accuracy of stereo matching. Recently, learning based stereo matching using CNN is widely researched, and we extended it to Multi-scale model in order to apply to underwater environment where scale variation of bubbles is large. Then, robust stereo matching can be done in bubbly environment. The code is available from here: Underwater Multi-scale CNN stereo
Network architecture of Multi-scale CNN stereo

Projected pattern removal by Multi-scale CNN

Captured images contain projected pattern from light source, but it is desireble to remove such pattern to acquire original texture. Since these projected pattern also has wide variation of scale, we used Multi-scale CNN to remove it.
Results of pattern removal by Multi-scale CNN

Real underwater environment measurement

We measured a live moving fish and swimming human body to confirm the feasibility of proposed method.
Resources
Publications
Kawasaki Laboratory