(4th most frequently cited paper in the computer graphics literature, according to Google Scholar)
A number of techniques have been proposed for flying through scenes by redisplaying previously rendered or digitized views. Techniques have also been proposed for interpolating between views by warping input images, using depth information or correspondences between multiple images. In this paper, we describe a simple and robust method for generating new views from arbitrary camera positions without depth information or feature matching, simply by combining and resampling the available images. The key to this technique lies in interpreting the input images as 2D slices of a 4D function - the light field. This function completely characterizes the flow of light through unobstructed space in a static scene with fixed illumination.
We describe a sampled representation for light fields that allows for both efficient creation and display of inward and outward looking views. We have created light fields from large arrays of both rendered and digitized images. The latter are acquired using a video camera mounted on a computer-controlled gantry. Once a light field has been created, new views may be constructed in real time by extracting slices in appropriate directions. Since the success of the method depends on having a high sample rate, we describe a compression system that is able to compress the light fields we have generated by more than a factor of 100:1 with very little loss of fidelity. We also address the issues of antialiasing during creation, and resampling during slice extraction.
Many people have commented on the striking similarity between this paper and the paper immediately following it in the Siggraph '96 proceedings [1]. Even the notations are similar. These similarities are not a coincidence, and they are easily explained.
The idea of light fields did not originate with these papers. As a footnote in our paper explains, this idea dates back to Arun Gershun's classic 1936 paper on the vector irradiance field [2] and Parry Moon's later papers on the scalar irradiance field, which he called the photic field [3]. The idea that the light field reduces to a 4D function in free space has been observed by many people (see our paper for citations). The idea that by extracting 2D slices from this 4D function one could construct new perspective views of a scene is ours, dating to a 1993 sketch in one of Marc Levoy's notebooks [4]. The ideas of parameterizing light fields using intersections of rays with two planes, and the relationship between light field antialiasing and the aperture of a camera, are also ours. (The latter idea also appears in the holography literature.) These ideas were communicated to the authors of the Lumigraph paper, who were already working on acquiring and representing arrays of images, during several chance meetings in March, April, August, and November of 1995. These meetings led them to pursue a similar development path. The ideas of using 3D geometric information during image reconstruction, and of building a light field from irregularly sampled input images, are theirs.
Happily, both papers were accepted for publication, and both make unique contributions. The communication between our two groups also explains the similarity in our notations: we used u,v and s,t to denote the camera and focal planes, respectively; they used s,t and u,v to denote these two planes. The acknowledgements section of the Lumigraph paper correctly attribute to us the ideas that we contributed.
By the way, like "light fields", the word "Lumigraph" has a long and colorful history. Around 1950, modern artist Oskar Fischinger invented a device for performing light shows that he called the Lumigraph. Accompanied by music, these shows were performed and filmed internationally by Fischinger and others. He named the device the "Lumigraph" in 1951, in collaboration with art critic Jules Langsner. An even earlier use of the term is due to George Hausmann, who was the first to bring motion picture shows to New Zealand. Around 1900, he toured under the name "Professor Hausmann's Lumigraph Company". (Thanks to David Koller for these historical notes.)
[1] Gortler, S.J., Grzeszczuk, R., Szeliski, R., Cohen, M., The Lumigraph, Proc. SIGGRAPH '96, ACM, 1986, pp. 43-54.
[2] Gershun, A., The Light Field, Moscow, 1936. Translated by P. Moon and G. Timoshenko in Journal of Mathematics and Physics, Vol. XVIII, MIT, 1939, pp. 51-151.
[3] Moon, P., Spencer, D.E., The Photic Field, MIT Press, 1981.
[4] Levoy, M., Rebinning pixels to yield perspective views, Stanford notebook no. 5, p. 64, May, 1993.
