Multiview Geometry and Self Calibration Revisited: A Differential Geometric Approach

Shankar Sastry
University of California, Berkeley

Abstract

Joint work with Ma Yi and Jana Kosecka.

In May 99, we presented at the ISL seminar at Stanford Structure and Motion Recovery from two frames: An optimization/estimation theoretic viewpoint by Yi Ma, Jana Kosecka and Shankar Sastry. Ths second talk in the AI-Vision-Robotics colloquium will be focused on Multiview Geometry and Self Calibration Revisited: A Differential Geometric Approach.

Multiview geometry has been traditionally developed in the framework of projective geometry. In this talk, we show an alternative approach which uses notation and concepts from differential geometry. We review projective (multilinear) constraints and Euclidean invariants associated with the problem of structure and motion recovery from n views. As a consequence of the study of projective constraints we show geometric dependency of the trilinear and quadrilinear constraints on the bilinear ones and associated conditions on motions which guarantee the dependency. The study of Euclidean invariants leads us to a new derivation and interpretation of Kruppa's equations as a inner product coinvariant of Euclidean transformations in a space with unknown metric. Our differential geometric approach allows us to establish the results in an elegant and concise way and reveal the intrinsic geometric meaning of some classic problems. New results and new algorithms fall naturally out of these new geometric interpretations.

The necessary and sufficient conditions for being able to estimate scene structure, motion and camera calibration from a sequence of images are very rarely satisfied in practice. What exactly, then, can be estimated in sequences of practical importance, when such conditions are not satisfied? In this talk we give a complete answer to this question. For every camera motion that fails to satisfy the conditions for unique reconstruction, we give an explicit characterization of the ambiguity in the reconstructed scene, motion and calibration. When the purpose of the reconstruction is to generate novel views of the scene, we characterize the vantage points that give rise to a valid Euclidean reprojection. We also characterize viewpoints that make the reprojection invariant to the ambiguity.

Biographical Information

Dr. Sastry received his B. Tech from the Indian Institute of Technology in Bombay in 1977. He received an M. A. in Mathematics in 1980 and Ph. D. in Electrical Engineering from the University of California, Berkeley in 1981 and a Master of Arts (honoris causa) from Harvard University in 1994. He taught at MIT in 1981-1982 and has been at Berkeley from 1983 to the current time. He is currently a Professor of Electrical Engineering and Computer Sciences and the Director of the Electronics Research Laboratory. He was a Gordon McKay Professor of Electrical Engineering and Computer Sciences at the Division of Applied Sciences, Harvard University in 1994, and a visiting Vinton Hayes fellow at MIT in the fall of 1992. He has held visiting positions at the Australian National University in Canberra, the Universita di Roma, Italy, the Scuola Normale in Pisa and the CNRS Laboratory LAAS in Toulouse, France. He is currently the Director of the Electronics Research Laboratory at Berkeley.

Shankar Sastry won the IEEE Region X Best student paper award in 1977, the President of India medal in 1977, the NSF Presidential Young Investigator award in 1985, the Eckman Award of the American Control Council in 1990. He is a Fellow of the IEEE. He is an Associate Editor of the IMA Journal of Mathematical Control and Information, the International Journal of Adaptive and Optimal Control and the new Journal of Sensors and Biomimetic Systems (Birkhauser), He has been an Associate Editor of the Journal of Mathematical Systems, Estimation and Control, IEEE Transactions on Automatic Control, the IEEE Transactions on Circuits and Systems, Large Scale Systems and the IEEE Control magazine.

He has authored papers in the areas of nonlinear control, the hierarchical organization of the control of complex systems, hybrid systems, adaptive control, robotics, unmanned aerial vehicles, nonholonomic mechanics and multi-fingered robot hands. Most recently, he has been interested in the following research areas:

  • millimeter scale robotics for surgery and simulation and visualization techniques for training surgeons.
  • Hybrid control systems involving a combination of discrete and continuous dynamics.
  • Distributed control of multi-agent control systems such as those arising in air traffic management and road transportation systems.
  • Unmanned Aerial Vehicles: sensor fusion, fault handling and coordinated control.
    Eyal Amir
    Last modified: Sat May 29 17:50:18 PDT 1999