Generating Virtual Worlds from Real Worlds using Computer Vision

Katsushi Ikeuchi

Carnegie-Mellon

Abstract

Virtual reality simulation (VRS) is a convenient tools for various applications: computer games, military simulation, and robot programming. Currently, most models for these systems have been manually constructed by programmers.

We are developing techniques to construct solid models from observation of real objects and scenes. This technique can bypass expensive programming efforts and can drastically reduce the cost of constructing such models. Since this technique maintains 3D solid models instead of a sequence of images (as in image mosaic systems), we can reduce the amount of memory and render the objects/scenes more accurately.

In this talk, I will present three key technologies recently developed in our group:

• SAI (spherical attribute image) for registration of unknown range images,
• PCAMD (principal component analysis with missing data) for obtaining the statistically optimal representation from a sequence of registered (yet noisy) range images, and
• PHOTOMETRIC SAMPLING for recovering reflectance characteristics of objects from a sequence of color images.

Finally, I will overview on-going projects including building a virtual Carnegie Mellon Wean Hall corridor using a movable range finder cart.