Shape From Light Field
Introduction
Light Field is a database of rays bounded by some parametric space. It describes the scene using a large number of rays, so that we could render photorealistic scenes in real time. In addition to real time rendering, light fields also encode a lot of information about the underlying 3D scene. In this research I would like to find out how much scene information, particulary the shape, could be reconstructed from a light field.
Possible applications:
- Shape assisted light field compression.
- Light field rendering with other objects.
- Non-intrusive high quality scene reconstruction.
- Next best view problem.
- Geometry based video and image compression.
- Explore the logic extremes of passive vision.
Flatland Light Fields
In current stage, I only experiment with synthesized flatland light fields for easy visualization. I move an virtual camera alone a circular path around the scene, and stack the 1D images into a volume, called flatland light field or epipolar plane image, the following shows the apparatus for the cubes scene:
I change the materials and textures of the cubes and balls, and get the following flatland light fields:
| Case 1 | Case 2 | Case 3 | Case 4 |
|
|
|
|
The description of the scenes:
| Scene | Description |
| 1 | The red cube is shiny, nothing is textured |
| 2 | The 2 balls are shiny, and everything is textured using peel texture except the white ball |
| 3 | The red cube is shiny, and everything is textured using peel texture except the white ball |
| 4 | The red cube is shiny, and everything is textured using regular texture except the white ball |
A similar appratus is set up for the buddha model:
I acquire the light field around the chest and skirt of the buddha, respectively:
| Chest | Skirt |
|
|
And I get the following flatland light fields for the buddha chest and skirt:
| Chest | Skirt |
|
|
Early Results
The following shows the reconstruction using my algorithm using only 120 directions(120 rows of the flatland light fields above), where:
- Color:
- Alpha scaled voxel color volume, each voxel is colored according to my reconstruction, and their color is alpha scaled, where alpha is the confidential value of that voxel.
- Weight:
- The occupancy mask of the scene, I transform the confidential values between 0 and 255. They are used as the alpha channels for color images.
- Reference:
- The true geometry of the scene.
| Scene | Cubes 1 | Cubes 2 | Cubes 3 | Cubes 4 | Buddha
Chest | Buddha
Skirt |
| Color
|
|
|
|
|
|
|
| Weight
|
|
|
|
|
|
|
| Reference
|
|
|
|
|
|
|
Conclusion:
The construction is pretty good, the only 2 cases the algorithm has problem is flat nontextured surfaces, like Cubes 1. Even if the surface has specular reflection, if it contains textures like the red cube in Cubes 3, we could still get pretty good reconstruction. The good reconstruction of the buddha scenes demonstrate that my algorithm indeed works for complex geometries.
liyiwei@graphics.stanford.edu
