Gigapixel Videography

Introduction

Gigapixel videography, beyond the resolution of single camera and human visual perception, aims to capture large-scale dynamic scene with extremely high resolution. Restricted by the spatial-temporal bandwidth product of optical system, the size, weight, power and cost are central challenges in gigapixel video. Aiming for the scalable, we developed a novel gigapixel videography system with unstructured multi-scale camera array design. ‘Unstructured’ indicates that the overall structure of this camera array does not follow fixed or particular designs thus without precise assembling and careful calibration in advance. `Multi-scale’ means not only the parameters of global-view camera varies from local-view camera, but also the parameters of local-view cameras can be different. The UnstructuredCam, an end-to-end unstructured multi-scale camera system, shows the ability of realtime capture, dynamically adjusting local-view cameras, and online warping for synthesizing gigapixel video. For the first time, this system shows the ability of capturing a seamless gigapixel video with adaptive and online pixel allocation for important or desired scene regions.

Highlights

Cross resolution image matching and warping

• A multi-scale camera array

• An iterative feature matching and image warping method

• The key feature of the proposed algorithm

• Robustness to and high accuracy for the huge resolution gap, camera parallaxes, complex scene appearances and color inconsistency among cameras

Yuan, X., Fang, L., Dai, Q., Brady, D. J., & Liu, Y. (2017, May). Multiscale gigapixel video: A cross resolution image matching and warping approach. In 2017 IEEE International Conference on Computational Photography (ICCP) (pp. 1-9). IEEE.

Real-time globally consistent dense 3D reconstruction

• Aiming at the practical usage of dense 3D reconstruction on portable devices

• FlashFusion: A Fast LArge-Scale High-resolution (sub-centimeter level) 3D reconstruction system

• Globally-consistent localization & spatial hashing

Han, L., & Fang, L. (2018). FlashFusion: Real-time Globally Consistent Dense 3D Reconstruction using CPU Computing. RSS.