Non-line-of-sight reconstruction with signal-object collaborative regularization

Non-line-of-sight (NLOS) imaging aims at recovering obscured objects from multiple scattered light. It has recently received widespread attention due to its potential applications such as autonomous driving, rescue operations, and remote sensing. In real applications, lasers or other light sources are used to illuminate a visible wall, the scattered light from which reaches the hidden object and is scattered back again. The photons collected by detectors can be used to recover the location, shape, albedo, and normal of the target. However, the measurements are unavoidably corrupted with noise, which is one of the major obstacles to get high-quality reconstructions. When the measurement noise is high, the targets reconstructed with existing methods are usually noisy with blurred boundaries.

In a new paper published in Light Science & Application, a team of scientists, led by Professor Xing Fu from Department of Precision Instrument, Tsinghua University, China, and Professor Lingyun Qiu from Yau Mathematical Sciences Center, Tsinghua University, China, have developed a unified framework for high-quality and noise-robust NLOS reconstructions. The technique is based on collaborative regularization of the signal and the reconstructed object, termed as the Signal-object collaborative regularization (SOCR) method.

Different from previous works that uses the raw measurements as input data directly, an approximation of the ideal signal is introduced in the SOCR framework. The regularization term designed focuses on the sparseness and non-local self-similarity of the hidden object as well as the smoothness of the estimated signal. This newly-developed framework is powerful in reconstructing both the albedo and the surface normal of the hidden targets under the general non-confocal settings. The reconstructions obtained have clear local structures, sharp boundaries, and little noise in the background, even in the presence of heavy noise in raw measurements. The reported method and technique will open new avenues for recognition and classification tasks in autonomous driving, rescue operations, and remote sensing in the future.

These scientists summarize the operational principle of their reconstruction framework:

"We design the collaborative regularization term under three assumptions: (1) The reconstructed target is sparse in the reconstruction domain; (2) Local structures of the hidden object repeat many times in the reconstruction domain; (3) The signal corresponding to the reconstructed target is smooth."

"The proposed framework can also be used as a plug-in module in different physical models. In addition, the proposed collaborative regularization term can be further simplified to accommodate cases where only the albedo needs to be reconstructed," they added.

"In the proposed collaborative regularization term, two dictionaries are used to capture the local structures and non-local correlations of the scene out of direct line of sight. The dictionary atoms and their corresponding coefficients can be viewed as features of the reconstructed target, which can be used for further tasks, such as recognition and classification in various applications," the scientists forecast.