Non-line-of-sight reconstruction with signal-object collaborative regularization

Non-line-of-sight reconstruction with signal-object collaborative regularization
(a), To reconstruct the unseen object, some light sources are used to illuminate a visible wall. The photons bounced back from the object are detected at several points on the visible wall. (b), Ground truth and our reconstruction. The x, y and z components are displayed in their absolute values. (c), Flowchart of the algorithm. The proposed regularization framework incorporates sparseness and non-local self-similarity of the hidden object as well as smoothness of the signal. Two orthogonal dictionaries are used to capture the local structures and non-local correlations of the hidden target. The estimated signal and the learned patterns of the target are shown on the left. Reconstructed albedo is shown on the right. Credit: Xintong Liu, Jianyu Wang, Zhupeng Li, Zuoqiang Shi, Xing Fu, Lingyun Qiu

Non-line-of-sight (NLOS) imaging aims at recovering obscured objects from multiple scattered light. It has recently received widespread attention due to its potential applications such as autonomous driving, rescue operations, and remote sensing. In real applications, lasers or other light sources are used to illuminate a visible wall, the scattered light from which reaches the hidden object and is scattered back again. The photons collected by detectors can be used to recover the location, shape, albedo, and normal of the target. However, the measurements are unavoidably corrupted with noise, which is one of the major obstacles to get high-quality reconstructions. When the measurement noise is high, the targets reconstructed with existing methods are usually noisy with blurred boundaries.

In a new paper published in Light Science & Application, a team of scientists, led by Professor Xing Fu from Department of Precision Instrument, Tsinghua University, China, and Professor Lingyun Qiu from Yau Mathematical Sciences Center, Tsinghua University, China, have developed a unified for high-quality and noise-robust NLOS reconstructions. The technique is based on collaborative regularization of the signal and the reconstructed object, termed as the Signal-object collaborative regularization (SOCR) method.

Different from previous works that uses the raw measurements as input data directly, an approximation of the ideal signal is introduced in the SOCR framework. The regularization term designed focuses on the sparseness and non-local self-similarity of the hidden object as well as the smoothness of the estimated signal. This newly-developed framework is powerful in reconstructing both the albedo and the surface normal of the hidden targets under the general non-confocal settings. The reconstructions obtained have clear local structures, sharp boundaries, and little noise in the background, even in the presence of heavy noise in raw measurements. The reported method and technique will open new avenues for recognition and classification tasks in autonomous driving, rescue operations, and remote sensing in the future.

These scientists summarize the operational principle of their reconstruction framework:

"We design the collaborative regularization term under three assumptions: (1) The reconstructed target is sparse in the reconstruction domain; (2) Local structures of the hidden repeat many times in the domain; (3) The signal corresponding to the reconstructed target is smooth."

"The proposed framework can also be used as a plug-in module in different physical models. In addition, the proposed collaborative regularization term can be further simplified to accommodate cases where only the albedo needs to be reconstructed," they added.

"In the proposed collaborative regularization term, two dictionaries are used to capture the local structures and non-local correlations of the scene out of direct line of sight. The dictionary atoms and their corresponding coefficients can be viewed as features of the reconstructed target, which can be used for further tasks, such as recognition and classification in various applications," the scientists forecast.

More information: Xintong Liu et al, Non-line-of-sight reconstruction with signal–object collaborative regularization, Light: Science & Applications (2021). DOI: 10.1038/s41377-021-00633-3

Journal information: Light: Science & Applications

Citation: Non-line-of-sight reconstruction with signal-object collaborative regularization (2021, September 27) retrieved 21 July 2024 from https://phys.org/news/2021-09-non-line-of-sight-reconstruction-signal-object-collaborative-regularization.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Non-line-of-sight imaging with picosecond temporal resolution

6 shares

Feedback to editors