--------------------------------------------------------------------------------------------------------------------

                          Rank Constraints for Space-Time Tracking and
                                   Non-Rigid Structure from Motion

                                         Lorenzo Torresani
                                         Stanford University

--------------------------------------------------------------------------------------------------------------------

We propose a novel solution for flow-based tracking and 3D reconstruction
of deforming objects in monocular image sequences. A non-rigid 3D object
undergoing rotation and deformation can be effectively approximated using
a linear combination of 3D basis shapes. From this representation we
derive a space-time rank constraint that allows us to redefine the
tracking problem. Instead of first training a model and then tracking the
model parameters, we are able to derive trajectory constraints first, and
then estimate the model. The bound on the rank is also exploited to
handle occlusion at the tracking level leading to the possibility of
recovering the complete trajectories of occluded/disoccluded points.
Following the same low-rank principle, the resulting flow matrix can be
factored to get the 3D pose, configuration coefficients, and 3D basis
shapes. The flow matrix is factored in an iterative manner, looping
between solving for pose, configuration, and basis shapes. The flow-based
tracking is applied to several video sequences and provides the input to
the 3D non-rigid reconstruction task. Additional results on synthetic
data and comparisons to ground truth complete the experiments.

This is joint work with Danny B. Yang, Eugene J. Alexander and 
Christoph Bregler.

Web page: http://movement.stanford.edu/nonrig/