Learning 3D Mesh Segmentation and Labeling

People

Abstract

This paper presents a data-driven approach to simultaneous segmentation and labeling of parts in 3D meshes. An objective function is formulated as a Conditional Random Field model, with terms assessing the consistency of faces with labels, and terms between labels of neighboring faces. The objective function is learned from a collection of labeled training meshes. The algorithm uses hundreds of geometric and contextual label features and learns different types of segmentations for different tasks, without requiring manual parameter tuning. Our algorithm achieves a significant improvement in results over the state-of-the-art when evaluated on the Princeton Segmentation Benchmark, often producing segmentations and labelings comparable to those produced by humans.

Paper

LabelMeshes.pdf, 4MB

Citation

Evangelos Kalogerakis, Aaron Hertzmann, Karan Singh, "Learning 3D Mesh Segmentation and Labeling", ACM Transactions on Graphics, Vol. 29, No. 3, July 2010 (also in SIGGRAPH 2010, Los Angeles, USA)

Bibtex

Presentation

LabelMeshes_web.ppt, 7MB
LabelMeshes_web.pdf, 4.4MB..
The above powerpoint and pdf files contain the Siggraph 2010 presentation of our method.
Note: You might need to install the Myriad Pro fonts to view the ppt file correctly.

Training Dataset

labeledDb.7z, 63.8MB.
This archive contains the dataset that we used for our experiments. The files contain the indices of faces belonging to each label per mesh. The meshes and segmentations come from the Princeton Segmentation Benchmark. See Section 5, "Data Set" paragraph of our paper for more details.

Results

results.7z, 3.8MB.
This archive contains segmentation and labeling results from one of the leave-one-out-error experiments. See Section 5, "Data Set" paragraph of our paper for more details.

Code

ShapeFeaturesCode.zip, 83KB.
This archive contains code for exporting the shape features included in x-tilde (see paper for details) and an implementation of Joint Boosting. The code uses Trimesh2 (version 2.8) and LAPACK and is part of a larger project, thus, it cannot be compiled as it is. This implementation is also not that optimized. For graph cuts in CRFs, I used the code provided here.

If you use this code or its parts in a research paper, please cite the above paper.

Acknowledgements

We thank Xiaobai Chen, Aleksey Golovinskiy, and Thomas Funkhouser for providing their segmentation benchmark and code, Szymon Rusinkiewicz for trimesh2, and Olga Veksler for the graph cut code. We thank Daniela Giorgi and AIM@SHAPE for providing the meshes from the Watertight Track of SHREC 2007 that are included in the benchmark. We also thank David Fleet and John Hancock for computing resources, and Olia Vesselova for proofreading. This project was funded by NSERC, MITACS, CFI, and the Ontario MRI. This work was done while Aaron Hertzmann was on a sabbatical visit at Pixar Animation Studios.