Seminar on Human, Avatar, and Robot Motion

Capturing human motion from a single camera without requiring markers, a suit or wearable sensors has been a widely studied problem in academia and industry. Approaches purely based on visual input ("RGB", e.g., a webcam) have failed to achieve the robustness and accuracy necessary for practical applications. With the advent of affordable depth-sensing cameras (the "D" in "RGB-D"), such as time-of-flight cameras or structured light stereo systems, a powerful new source of information about the analyzed scene becomes available. In this talk, we will lay out the basic set of tools required to analyze RGB-D data and discuss alternative ways of using them to solve the human motion capture problem.

Humans, animals, robots, and physics-based characters all need motor skills to interact with the world that surrounds them. Yet general methods for developing control strategies for skilled movement and manipulation remain elusive. In this talk we will take stock of recent advances in biped and quadruped locomotion with an eye towards identifying features of motor control representations that support the scalable development of motor skills and that allow them to generalize well. We will argue in favour of:
(1) incremental learning in several different guises;
(2) task-specific abstraction of state and dynamics; and
(3) hybrid force and position control.
We will illustrate these examples with simulated bipeds and quadrupeds that are capable of a broad range of locomotion-related skills. This is joint work with many of my recent students and postdocs.

I will present our ongoing work on human simulation, specifically (1) the comprehensive biomechanical simulation of the human body, confronting the challenge of modeling and controlling more or less all of the relevant articular bones and skeletal muscles, as well as simulating the physics-based deformations of the soft tissues, and (2) an artificial life framework for multi-human simulation that addresses the problem of realistically emulating the rich complexity of human activity in urban environments, yielding 3D virtual worlds populated by lifelike autonomous pedestrians. Among their many uses, our advanced human models are showing promise in applications to Computer Vision, including the study of active sensorimotor learning and neuromuscular control, as well as within a paradigm that we call "Virtual Vision", in which virtual reality -- specifically 3D virtual worlds populated by lifelike autonomous pedestrians -- subserves computer vision research for the purposes of persistent surveillance by smart camera sensor networks. I will also elucidate the profound scientific and computational challenges that remain in spanning physics and AI/ALife for the purposes of tractable, lifelike human simulation.

At some point in the future, robots will be taught new skills in a similar way as we teach skills to children, starting from demonstrating a behavior up to autonomous self-improvement with trial-and error learning. Our research is concerned with how to bootstrap such motor skills. We start with a general representation of motor skills based on learnable nonlinear attractor landscapes. Skills can be initialized by imitation learning or general planning methods.
For autonomous skill improvement, we discovered a new trajectory-based reinforcement learning method, call PI2 (Policy Improvement with Path Integrals). PI2 is based on stochastic optimal control and approximation methods developed in particle physics. The algorithmic form of PI2 is surprisingly simple, and lends itself to both model-based and model-free learning scenarios. Cost functions can have arbitrary state-dependent cost without the requirement of differentiability -- only the command cost is assume to be quadratic.
Learning with hidden states is possible, too. We will demonstrate several synthetic and robotic results that demonstrate that PI2 scales to learning in high dimensional motor systems, that it outperforms previous model-free algorithms in reinforcement learning in accuracy and learning speed, and that it can be applied to topics like learning in stochastic manipulation and variable impedance control. As PI2 has only one open tuning parameter, the exploration noise, it can be considered a significant step forward towards an efficient black-box reinforcement learning approach for high-dimensional motor systems.

Inverse reinforcement learning (IRL) aims to infer goals, rewards, and cost functions from observations of an agent behaving according to an optimal policy. IRL algorithms have been used to learn robot walking motions from demonstration, build route navigation algorithms that mimic human drivers, and even to learn about how real people infer the goals of abstract agents. In this talk, I will provide a detailed tutorial on notable inverse reinforcement learning algorithms. I will then discuss preliminary ideas for applying such techniques to analyzing and reproducing human motion.

The talk will touch the latest industry developments in the realm of 3D character animation for games and how what was a niche art is becoming more and more a mainstream phenomenon. The challenges in the process of democratization of 3D animation will be presented as a starting point for a deeper discussion on research solutions and mathematical methods for the generation of 3D characters and animations. The main topics will include animation generative models, motion graphs, automatic skinning of 3D meshes, automatic rigging of characters, animation retargeting.

In this talk, I will provide an overview of Laban Movement Analysis (LMA).
LMA is a leading framework for describing, annotating and analyzing expressive aspects of movement. The system is used in a wide range of applications, from dance annotation, to performance training, to somatic therapy. I will briefly discuss the four main components of LMA - Body, Effort, Shape and Space - and elaborate on those aspects most useful for character animation. Time permitting, I will provide an overview of some of the computer animation work based on the system and discuss challenges in applying the system in a computational framework.

I will discuss an important subproblem in physics-based animation --- controller synthesis for humanoid characters. We describe a method for optimizing the parameters of a physics-based controller for full-body, 3D walking. The objective function includes terms for power minimization, angular momentum minimization, and minimal head motion, among others. Together these terms produce a number of important features of natural walking, including active toe-off, near-passive knee swing, and leg extension during swing.
We then extend the algorithm to optimize for robustness to uncertainty.
Many unknown factors, such as external forces, control torques, and user control inputs, cannot be known in advance and must be treated as uncertain. Controller optimization entails optimizing the expected value of the objective function, which is computed by Monte Carlo methods.
We demonstrate examples with a variety of sources of uncertainty and task constraints.
Joint work with David Fleet and Aaron Hertzmann, University of Toronto
Note: Some people might have already seen this talk at a NMBL meeting in August.

Understanding the control forces that drive humans and animals is fundamental to describing their movement. Although physics-based methods hold promise for creating animation, they have long been considered too difficult to design and control. Likewise, physical motion models, if developed, could be very valuable to human pose tracking in computer vision. I will outline the main problems of human motion modeling. I will then present a new approach to control of physics-based characters based on high-level features of human movement. These controllers provide unprecedented flexibility and generality in real-time character control: they capture many natural properties of human movement, they can be easily modified and applied to new characters, and they can handle a variety of different terrains and tasks, all within a single control strategy. Until very recently, even making a controller walk without falling down was extraordinarily difficult. This is no longer the case. Our work, together with other recent results in this area, suggests that the research community is ready to make great strides in locomotion. Joint work with Martin de Lasa and Igor Mordatch.

Digital human modeling (DHM) has emerged as a useful tool for virtual prototyping, ergonomic analysis, and virtual training. Inserting a digital human into a simulation or virtual environment, DHM can facilitate the validation of product design and the prediction of performance and safety in the processes of assembly and maintenance. The current commercially available DHM packages such as JACK and SantosHuman, however, have limited capabilities on generating realistic human motions in complex environments. In this talk, I will describe new algorithms for planning and synthesizing human motions for DHM. Based on the fact that a human model is a tightly coupled system, we first develop an efficient sampling-based planner that coordinates the motion between different human body parts and computes feasible paths for tight environment. We then develop a new motion blending algorithm that refines the path computed by the planner with motion capture data to compute a smooth and plausible trajectory. We demonstrate the performance of our algorithms on a 40-DOF articulated human model and generate efficient motion strategies in complex CAD environments. Finally, I will suggest new research directions on generating human motions for DHM and virtual environment.

Legged robots offer the potential to navigate terrain that is completely inaccessible to wheeled vehicles. In this talk I will describe our development of a software architecture for a quadruped robot, capable of climbing over a wide variety of challenging, previously unseen terrains. The system uses both slow "static" walking for highly complex, rocky terrains, and faster "dynamic" maneuvers and gaits for quickly crossing less irregular terrains. New machine learning and new control techniques have played a crucial role in developing this system, and I'll describe key components where we benefitted from such algorithms.