Beyond point-to-point triangulation: Dynamic 3D imaging under global illumination

As the "eyes of industry," 3D measurement technology provides comprehensive, accurate, and reliable 3D information for high-end precision manufacturing. Optical measurement methods, such as structured light projection, have been widely adopted in various fields, including industrial inspection, intelligent assembly, advanced manufacturing, and material sorting.

These methods, based on the triangulation principle, rely on the traditional geometric optics concept of "point-to-point" imaging. Consequently, they are typically limited to scenarios with direct illumination conditions.

As manufacturing processes evolve and smart application scenarios multiply, the demands on measurement tasks are shifting from simple static 3D measurements in straightforward environments to high-precision dynamic measurements in complex mixed environments. In industrial inspection and intelligent assembly, for example, measurements often involve various reflective surfaces.

Challenges include multi-path indirect illumination caused by inter-reflections on metallic or mirror-like surfaces, and subsurface scattering in mixed surfaces such as jade, ice, skin, and wax. Similarly, advanced manufacturing and material sorting require dynamic 3D measurements through complex transmission media.

For instance, real-time monitoring of the forming state in metal additive 3D printing must contend with volumetric scattering indirect illumination, while rapid identification and classification of objects within translucent surfaces in material sorting involves multi-path direct/indirect illumination, as illustrated in fig. 1. All these challenging illumination conditions are collectively referred to as global illumination.

Under global illumination, light propagation paths disrupt the "point-to-point" single reflection conditions required by traditional imaging models. The direct application of conventional 3D measurement methods may fail due to the mixing and/or loss of depth-encoded information, significantly limiting the applicable scenarios for existing measurement techniques.

Multi-scale parallel single-pixel imaging

To this end, a Fourier dual-slice projection with depth-constrained localization is presented to separate and utilize different illumination and reflection components efficiently, which can significantly decrease the number of projection patterns in each sequence from thousands to 15, as shown in fig. 2(a).

Subsequently, multi-scale parallel single-pixel imaging (MS-PSI) is proposed based on the proposed and proven position-invariant theorem, which breaks the local regional assumption and enables dynamic 3D reconstruction, as shown in fig. 2(b).

Results

Unlike the traditional structured-light method, the proposed MS-PSI method is no longer limited by the point-to-point triangulation rule and breaks existing PSI methods' regional light reception assumption. It provides an efficient and general solution for separating complex illumination information, including subsurface scattering, inter-reflection, multi-layer superpositions, and thin volumetric scattering media, by recording the four-dimensional light transport coefficient.

The experimental results pave the way for dynamic 3D reconstruction under complex optical field reflection and transmission conditions, benefiting imaging and sensing applications in advanced manufacturing, autonomous driving, and biomedical imaging.