| Non-Line-Of-Sight (NLOS) imaging aims at recovering the 3D geometry of objects that are hidden from the direct line of sight. One major challenge with this technique is the weak available multibounce signal limiting scene size, capture speed, and reconstruction quality. To overcome this obstacle, we introduce a multipixel time-of-flight non-line-of-sight imaging method combining specifically designed array detectors with a novel fast reconstruction algorithm that for the first time captures and reconstructs live real-time videos of non-line-of-sight scenes with natural non-retroreflective objects. We develop a model of the signal-to-noise-ratio of non-line-of-sight imaging and use it to devise a method that reconstructs the scene such that signal-to-noise-ratio, motion blur, angular resolution, and depth resolution are all independent of scene depth suggesting that reconstruction of very large scenes may be possible. |
| Real-time NLOS virtual image processing pipeline. The imaging system sends the virtual Phasor Field (PF) signal to the visible wall and captures the signal returning from the hidden scene back to the wall. The massive raw photon stream is recorded by the SPAD array. Raw photons from all channels are virtually remapped into a complete aperture. Then the remapped data is transformed into the frequency domain (Fourier Domain Histogram, FDH) and propagated by the fast Rayleigh Sommerfeld Diffraction (RSD) algorithm. Lastly, temporal frame averaging yields constant SNR throughout the entire reconstructed volume, and the result is displayed. |
To demonstrate the effectiveness of our method, we provide a baseline comparison with other methods. Besides the two 16×1 non-confocal SPAD arrays, our imaging system has a single pixel gated SPAD that can be used for confocal data acquisition. We collect data using confocal and non-confocal system with different exposure time and compare reconstruction quality.
The figure shows the comparison which is split into two parts, non-confocal and confocal data. The non-confocal data are used for direct reconstruction with the proposed method, and are also approximately converted to confocal data with the method described in Lindell et al.[1] , as previously also demonstrated in [2]. Then, the LCT [3] (“approximate LCT”) and FK migration [1] (“approximate FK”) are applied to this approximately confocal data. The reconstruction results of approximate LCT and approximate FK look blurry, have a hazy background, and look noisy at short exposure times. Our method successfully reconstructs the scene even with a short exposure time of 0.2s. After 1-4 seconds of exposure the reconstruction quality doesn’t change noticeably, which suggests our proposed method doesn’t require long exposure times. Previously Liu et al. [4] demonstrated that a Phasor Field reconstruction stabilizes at a certain exposure level and adding further exposure time does not affect the reconstruction.
Using the confocal measurements we reconstruct the scene using the confocal Phasor Field reconstruction version[2], LCT[3] and FK[1]. The confocal acquisition method in general suffers from noise with short exposure times and requires a long exposure time to achieve high quality reconstruction of diffuse targets. Therefore, often times confocal methods use retroreflective materials to increase the signal. Since the confocal measurement requires a dense scan of the relay wall, a confocal scan at 0.2 seconds was not possible with our galvo mirrors. Note that the times reported here are in line with what was reported in the original papers.
