A number of techniques have been proposed for flying through scenes by redisplaying previously rendered or digitized views. Apple's QuickTime VR is one example. Techniques have also been proposed for interpolating between views by warping input images, using depth information or correspondences between multiple images. The general notion of generating new views from pre-acquired imagery is called image-based rendering.
We are investigating a new image-based rendering technique that does not require depth information or image correspondences, yet allows full freedom in the range of possible views. The idea behind the technique is a representation of the light field - radiance as a function of position and direction in regions of space free of occluders (free space). There are two common situations where this representation suffices: when viewing an object from outside its convex hull, or when viewing an environment from a empty region such as the interior of a room. In free space, the light field is a 4D, not a 5D function. Creating a light field from a set of images corresponds to inserting 2D slices into the 4D light field representation. These images may be of virtual or real scenes, i.e. they may be rendered or digitized. The latter can be acquired using a video camera mounted on a computer-controlled gantry or using a hand-held camera. Once a light field has been created, new views may be constructed in real time by extracting 2D slices in appropriate directions.
Some key technical challenges in this area include finding appropriate parameterizations for the light field, creating light fields efficiently and with proper prefiltering, displaying them at interactive rates and with proper resampling, compressing them to reduce memory use, and building devices and systems for digitizing light fields from real scenes.
A list of technical papers, with abstracts and pointers to additional information, is also available. Or you can return to the research projects page or our home page.