Automatic Multiperspective Images

Augusto Román Stanford University		Hendrik P. A. Lensch Stanford University
To appear in the Proceedings of EGSR 2006

Multiperspective images generated automatically from a videostream with lateral movement
(a) A pushbroom image, which uses the perspective provided by the input stream in the y-direction and an orthographic projection in x, in order to combine the information of all frames. Notice the difference in perspective in x and y leads to severe distortion at the intersection and the alleyway. (b) A multiperspective image generated automatically using our technique. While the perspective in y is still the same, we optimized the perspective in x in each image segment in order to minimize distortion. Notice that this is an image with multiple perspectives -- there is a vanishing point down the alleyway and a separate vanishing point down the intersection.

Paper

High resolution images version - Adobe Acrobat PDF (12.0 MB)
Low resolution images version - Adobe Acrobat PDF (4.7 MB)

Presentation

Main presentation file - Powerpoint (19.0 MB)
ActiveX Object to display 3D animations - OCX file (216 KB)

NOTE: The ActiveX control must be registered with your system before it can be used. Two very simple batch files are provided for that: register_control.bat and unregister_control.bat Also, Powerpoint will ask, when opened, whether you would like to initialize the control with stored settings, to which you should answer "Yes" (if you trust me).

Abstract

Multiperspective images generated from a collection of photographs or a videostream can be used to effectively summarize long, roughly planar scenes such as city streets. The final image can span a larger field of view than any single input image. However, common projections used to make these images, including the cross-slits and pushbroom projections, may suffer from depth-related distortions in non-planar scenes. In this paper, we propose a metric for evaluating the distortion in these images due to deviation from the standard perspective projection. Minimizing this error metric we can automatically define the picture surface and viewpoints of a multiperspective image to reduce the distortion artifacts. This optimization requires only a coarse estimate of scene geometry, which can be provided as a depth map or in the form of a 2D spatial importance map defining interesting parts of the scene. These maps can be automatically constructed in many cases, allowing rapid generation of images of very long scenes.

Long result

Above is a horizontally compressed version of the longest panorama that we created using the automatic optimization. This panorama spans nearly 2 km of 18th Avenue in San Francisco. Notice how the profile of the image matches the hill along the street. Click on the link above to see a higher resolution version of the entire street. Below is a segment showing 1/10th of the full street. Click on that image to see the full resolution segment.