Broad Area Colloquium For AI-Geometry-Graphics-Robotics-Vision
(CS 528)

Dense Multiview Stereo Approaches for Handling Occlusions, Highlights, Reflections, and Translucency

Sing Bing Kang
Vision Technology Group
Microsoft Research Monday, Oct. 27, 2003, 4:15PM
TCSeq 200


While stereo matching was originally formulated as the recovery of 3D shape from a pair of images, it is now generally recognized that using more than two images can dramatically improve the quality of the reconstruction. Unfortunately, as more images are added, the prevalence of semi-occluded regions (pixels visible in some but not all images) also increases. In addition, non-rigid effects in real scenes such as highlights, reflections, and translucency further complicate multiview stereo.

In this talk, I will describe how we progressively tackle the problems of occlusion, highlights, reflections, and translucency. To handle occlusion, we use a combination of shiftable windows and a dynamically selected subset of the neighboring images to do the matches. To handle highlights, we apply a color histogram differencing technique. Finally, to take into account reflections and translucency, we model the image formation as additive superposition of two layers at two different depths, and solve for them iteratively. I will show results for both synthetic and real image sequences as validation of these approaches.

About the Speaker

Sing Bing Kang received his Ph.D. in robotics from CMU in 1994. He is currently a researcher at Microsoft Corporation, where he is working on environment modeling from images. His paper on the Complex Extended Gaussian Image had won the IEEE Computer Society Outstanding Paper award at CVPR'91. His IEEE Transactions on Robotics and Automation paper on human-to-robot hand mapping had been awarded the 1997 King-Sun Fu Memorial Best Transaction Paper award. Sing Bing has published about 20 refereed journal papers and about 45 refereed conference papers, and holds 11 US patents. He has also co-edited a book on panoramic vision, which was published by Springer in 2001.


Back to the Colloquium Page