Motion Capture

Introduction

Human motion capture has been a challenging research topic in computer vision and computer graphics for decades. The goal is to reconstruct a temporally coherent representation of the dynamically deforming surface of human characters from videos. A fast, low cost, easy to set up and promising for popularization systems is the long-term pursuit.

Highlights

Free-viewpoint human performance capture with dynamic details

• Multi-layer Avatar Digitization

• Physics-Based Performance Capture

Yu, T., Zheng, Z., Zhong, Y., Zhao, J., Dai, Q., Pons-Moll, G., & Liu, Y. (2019). SimulCap: Single-View Human Performance Capture with Cloth Simulation.. To appear in CVPR 2019

Real-time motion capture by double fusion

• Double-layer Representation

• Joint Motion Tracking

• Volumetric Fusion and Optimization

Yu, T., Zheng, Z., Guo, K., Zhao, J., Dai, Q., Li, H., … & Liu, Y. (2018). Doublefusion: Real-time capture of human performances with inner body shapes from a single depth sensor.. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 7287-7296).

Handheld-device enabled motion capture

• 3D Point Correspondences and Implicit RGBZ Point Cloud
  Segmentation

• Feature-Based Background Correspondences

• Optimization of Skeleton and Camera Poses

• 

• Non-rigid Mesh Deformation and Vertex Color Update

Ye, G., Liu, Y., Hasler, N., Ji, X., Dai, Q., & Theobalt, C. (2012, October). Performance Capture of Interacting Characters with Handheld Kinects. In European Conference on Computer Vision (pp. 828-841).

Markerless motion capture

• Multi-view Image Segmentation

• Pose Tracking and Surface Estimation

Liu, Y., Stoll, C., Gall, J., Seidel, H. P., & Theobalt, C. (2011, June). Markerless Motion Capture of Interacting Characters Using Multi-view Image Segmentation. In CVPR 2011 (pp. 1249-1256).

Research Production

Related studies were published on international conferences including CVPR(oral), ICCV, ECCV and SIGGRAPH, international journals including T-PAMI and T-VCG.