Reconstructing a high-accuracy absolute phase map for single-shot, unambiguous 3D surface imaging

A new publication from Opto-Electronic Advances discusses how deep learning makes single-shot, high-resolution structured light 3D imaging possible.

With the rapid development of optoelectronic information technology, three-dimensional (3D) imaging and sensing has become a research forefront in optical metrology. Following the transition from monochrome to color, from low-resolution to high-resolution, and from static photo to dynamic video, the evolution from 2D photography to 3D sensing has become the fourth imaging revolution. In this regard, fringe projection profilometry (FPP) is one of the most representative 3D imaging technologies due to its non-contact, high-resolution, high-speed, and full-field measurement capability, which has been widely applied in multiple fields, such as intelligent manufacturing, reverse engineering, industrial inspection, and heritage preservation.

In recent years, with the rapid advances of optoelectronic devices (e.g., digital light projectors, spatial light modulators, and high-speed image sensors) and digital signal processing units (e.g., high-performance computers and embedded processors), people subsequently set higher expectations on FPP: it should be both high precision and high speed. While these two aspects seem contradictory in nature, speed has gradually become a fundamental factor that must be taken into account when using FPP. For many fields, such as industrial manufacturing, speed means efficiency, and efficiency represents productivity. Under these circumstances, high-precision 3D reconstruction using only one single pattern has been the ultimate goal of structured light 3D imaging in perpetual pursuit.

The authors of this article report a deep-learning-enabled dual-frequency-multiplexing FPP technique, enabling single-shot unambiguous high-precision structured light 3D imaging. The proposed method constructed two parallel U-shaped networks with the same structure, one of which takes the well-designed dual-frequency composite fringe image as the network input, combined with the traditional phase-shifting physical model to predict the sine and cosine terms used to calculate the high-quality wrapped phase map, and the other is designed to predict the fringe order information from the input dual-frequency composite fringe image. Through learning from a large number of data sets, the properly trained neural networks can de-multiplex high-resolution, spectrum-crosstalk-free phases from the composite fringe and directly reconstruct a high-accuracy absolute phase map for single-shot, unambiguous 3D surface imaging.

The proposed deep-learning-enabled dual-frequency multiplexing FPP technique is expected to fill the speed gap between 3D imaging and 2D sensing, achieving true single-frame, high-precision, unambiguous 3D shape reconstruction. This research opens new avenues for single-shot, instantaneous 3D shape measurement of discontinuous and/or mutually isolated objects in fast motion. Nowadays, deep learning technology has fully permeated into almost all tasks of optical metrology. Despite the promising, in many cases impressive, results that have been reported in the literature, the underlying reasons behind these successes remain unclear at this stage.

Many researchers are still skeptical and maintain a wait-and-see attitude towards its applications in high-risk scenarios, such as industrial inspection and medical care. But it can be envisaged that with the further development of artificial intelligence technology, the continuous improvement of computer hardware performance, and the further development of optical information processing techniques, these challenges will gradually be solved soon. Deep learning will thus play a more significant role and make a more far-reaching impact in optics and photonics.