Femtosecond imaging

Introduction

Conventional digital photography usually assumes unlimited light speed and follows the principle of lens imaging, so the SBP and optical resolution of the camera is limited by the imaging lens. In ultra fast imaging, we proposed ‘Femtosecond laser + picosecond streak camera’ lensless imaging mechanism, transient imaging approach in frequency domain, and decoupling approach of transient global light transport matrix. The above setups and corresponding reconstruction methods broke the long-held assumption in photography that light travels in straight lines at infinite high speed, achieved recording of the photon traveling and parsing of the scene surface reflection properties. This imaging technique has great potential in biomedical imaging, material and component analysis in ultra-thin mobile device, remote sensing, and surveillance.

Highlights

Ultra-fast Lensless Computational Imaging through 5D Frequency Analysis of Time-resolved Light Transport

Light transport has been analyzed extensively, in both the primal domain and the frequency domain. Frequency analyses often provide intuition regarding effects introduced by light propagation and interaction with optical elements; such analyses encourage optimal designs of computational cameras that efficiently capture tailored visual information. However, previous analyses have relied on instantaneous propagation of light, so that the measurement of the time dynamics of light–scene interaction, and any resulting information transfer, is precluded. In this paper, we relax the common assumption that the speed of light is infinite. We analyze free space light propagation in the frequency domain considering spatial, temporal, and angular light variation. Using this analysis, we derive analytic expressions for information transfer between these dimensions and show how this transfer can be exploited for designing a new lensless imaging system. With our frequency analysis, we also derive performance bounds for the proposed computational camera architecture and provide a mathematical framework that will also be useful for future ultra-fast computational imaging systems.

Wu, D., Wetzstein, G., Barsi, C., Willwacher, T., Dai, Q., & Raskar, R. (2014). Ultra-fast lensless computational imaging through 5D frequency analysis of time-resolved light transport. International journal of computer vision, 110(2), 128-140.

Robust and accurate transient light transport decomposition via convolutional sparse coding

Ultrafast sources and detectors have been used to record the time-resolved scattering of light propagating through macroscopic scenes. In the context of computational imaging, decomposition of this transient light transport (TLT) is useful for applications, such as characterizing materials, imaging through diffuser layers, and relighting scenes dynamically. Here, we demonstrate a method of convolutional sparse coding to decompose TLT into direct reflections, inter-reflections, and subsurface scattering. The method relies on the sparsity composition of the time-resolved kernel. We show that it is robust and accurate to noise during the acquisition process.

Hu, X., Deng, Y., Lin, X., Suo, J., Dai, Q., Barsi, C., & Raskar, R. (2014). Robust and accurate transient light transport decomposition via convolutional sparse coding. Optics letters, 39(11), 3177-3180.

Frequency-Domain Transient Imaging

A transient image is the optical impulse response of a scene, which also visualizes the propagation of light during an ultra-short time interval. In contrast to the previous transient imaging which samples in the time domain using an ultra-fast imaging system, this paper proposes transient imaging in the frequency domain using a multi-frequency time-of-flight (ToF) camera. Our analysis reveals the Fourier relationship between transient images and the measurements of a multi-frequency ToF camera, and identifies the causes of the systematic error-non-sinusoidal and frequency-varying waveforms and limited frequency range of the modulation signal. Based on the analysis we propose a novel framework of frequency-domain transient imaging. By removing the systematic error and exploiting the harmonic components inside the measurements, we achieves high quality reconstruction results. Moreover, our technique significantly reduces the computational cost of ToF camera based transient image reconstruction, especially reduces the memory usage, such that it is feasible for the reconstruction of transient images at extremely small time steps. The effectiveness of frequency-domain transient imaging is tested on synthetic data, real data from the web, and real data acquired by our prototype camera.

Lin, J., Liu, Y., Suo, J., & Dai, Q. (2017). Frequency-domain transient imaging. IEEE transactions on pattern analysis and machine intelligence], 39(5), 937-950.

Scattering robust 3D reconstruction via polarized transient imaging

Reconstructing 3D structure of scenes in the scattering medium is a challenging task with great research value. Existing techniques often impose strong assumptions on the scattering behaviors and are of limited performance. Recently, a low-cost transient imaging system has provided a feasible way to resolve the scene depth, by detecting the reflection instant on the time profile of a surface point. However, in cases with scattering medium, the rays are both reflected and scattered during transmission, and the depth calculated from the time profile largely deviates from the true value. To handle this problem, we used the different polarization behaviors of the reflection and scattering components, and introduced active polarization to separate the reflection component to estimate the scattering robust depth. Our experiments have demonstrated that our approach can accurately reconstruct the 3D structure underlying the scattering medium.

Wu, R., Suo, J., Dai, F., Zhang, Y., & Dai, Q. (2016). Scattering robust 3D reconstruction via polarized transient imaging. Optics letters, 41(17), 3948-3951.