Image-Based Reconstruction of Spatially Varying Materials (BRDFs)

Abstract

The use of realistic models for all components of images synthesis is a fundamental prerequisite for photorealistic rendering. The generation of these models in a manual process often becomes infeasible as the demand for visual complexity increases steadily. In this sketch we concentrate on the acquisition of realistic materials. In particular, we describe an acquisition method for shift-variant BRDFs, i.e., a specific BRDF for each surface point.

Data Acquisition

We acquire the geometry of the object by use of a 3D scanner, e.g. a structured light or computer tomography scanner, yielding a triangular mesh. In order to capture the reflection properties we take a relatively small number (around 20) of high-dynamic range (HDR) images of the object, lit by a point-light source. We recover the camera position and orientation as well as the light source position relative to the geometric model for all images.

For every point on the object's surface we collect all available data from the different views in a data structure called {\it lumitexel}. It contains the position of the surface point, its normal, and a list of radiance samples together with their viewing and lighting directions.

Clustering of Materials

Since a single lumitexel does not carry enough information to reliably fit a BRDF model to the radiance samples, we first determine clusters of lumitexels belonging to similar materials. Starting with a single cluster containing all lumitexels, the parameters of an average BRDF are fitted using the Levenberg-Marquardt algorithm. From this, two new sets of parameters are generated by varying the fitted parameters along the direction of maximum variance, yielding two slightly separated BRDFs. The lumitexels of the original cluster are then assigned to the nearest of these BRDFs, forming two new clusters. A stable separation of the materials in the clusters is obtained by repeatedly fitting BRDFs to the two clusters and redistributing the original lumitexels. Further splitting isolates the different materials until the number of clusters matches the number of materials of the object as illustrated in the following figure.

The clustering process at work. In every image a new cluster was created.
The object was reshaded using the BRDF that was fitted to each cluster.

Spatially Varying Materials

After the clustering we still have the same reflection behavior assigned to all lumitexels in one cluster. However, small features on the surface and smooth transition between materials can only be represented if every lumitexel is assigned its own BRDF.

In our algorithm, this BRDF is a linear combination of the BRDFs recovered by the clustering procedure. This can be represented by a set of basis BRDFs for the entire model plus a set of weighting coefficients for each lumitexel. An optimal set of weighting coefficients minimizes the error between the measured radiance and the weighted radiance values obtained by evaluating the basis BRDFs for the specific viewing and lighting directions. To recover the coefficients we compute the least square solution of the corresponding system of equations using singular value decomposition.

This method allows for accurately shaded, photorealistic rendering of complex solid objects from new viewpoints under arbitrary lighting conditions with relatively small acquisition effort.

Here some pictures

BirdModel

Left: Last result of the clustering step. Right: Bird with the spatially varying BRDF determined by projecting each lumitexel into a basis of BRDFs. Note the subtle changes of the materials making the object look realistic.

Angel Model

Left side: Photograph of model. Right side: Model with acquired BRDF rendered from the same view with similar lighting direction. The difference in the hair region is due to missing detail in the triangle mesh.

Bust Model

A bronze bust rendered with a shift-variant BRDF, which was acquired with our reconstruction method.

Here some movies

Klick on the images to view the movies.

Literature

Hendrik P. A. Lensch, Jan Kautz, Michael Goesele, Wolfgang Heidrich, and Hans-Peter Seidel.
Image-Based Reconstruction of Spatially Varying Materials. In Rendering Techniques '01 (Proceedings of Eurographics Rendering Workshop), (accepted for publication).

Last modified: Thu Aug 26 14:13:11 PDT 2004 by ypcat: can't get local yp domain: Local domain name not set