Last updated June 2006
One of the great pleasures of being at Stanford is interacting with its talented and enthusiastic faculty, staff, and students. Described below are some of the research projects we are working on together. Aside from publishing technical papers about these projects, we often package and release our software and our data.
Measurement technologies for graphics. I like building things. Over the past 15 years, my students and I have built devices for measuring 3D shape, light fields, and reflectance functions. These particular devices all included a mechanical component. However, the trend is towards optoelectronic solutions. With this in mind, we recently developed a real-time range scanner based on video projectors, a handheld camera for capturing instantaneous light fields, a multi-camera array for acquiring video light fields, and a "light field microscope" for capturing light fields of tiny biological specimens. The latter two projects were collaborations with Pat Hanrahan, Mark Horowitz, and their students.
Light fields and related ideas. A light field is a 2D array of (2D) images, each taken from a different viewpoint. The resulting 4D array completely characterizes the passage of light through unoccluded space. By assembling pixels from several images, new views can be constructed from observer positions not present in the original array. This idea was described by Pat Hanrahan and myself in a 1996 Siggraph paper. The Stanford multi-camera array (mentioned above) has given us a unique device for exploring applications of light fields, including high-speed video using a dense camera array. Recently, we've been playing with two optical effects: synthetic aperture photography, in which we combine closely spaced views of an object, thereby letting us see through partially occluding objects like foliage, and synthetic aperture illumination, in which we physically project multiple images onto a common plane in space, thereby creating a synthetic image with an extremely shallow depth of field. Here's a paper about calibrating our camera array for synthetic aperture photography, here are some examples of seeing through people, and here's an array of miniature video projectors we're building to further explore synthetic aperture illumination. Using these two optical effects, we have implemented a discrete approximate of confocal imaging, a technique borrowed from microscopy. This approximation lets us selectively image or illuminate one object in a complex scene, and it lets us see further through scattering environments (such as turbid water) than is otherwise possible. We've also been exploring a technique we call dual photography, which uses Helmholtz reciprocity to swap the cameras and light sources in a scene. This technique allows us to produce images from viewpoints in the scene where a camera never stood. In another project, we've been exploring applications for the handheld light field camera (mentioned above) built by PhD student Ren Ng. Foremost among these applications is the ability to refocus a photograph after it has been taken. This dramatic effect must be seen to be believed; check out the video on this web page, or the web page of Ren's startup company, Refocus Imaging. By the way, Ren's PhD thesis won the 2006 ACM Doctoral dissertation Award. Recently, we adapted this idea to microscopy, by inserting a microlens array into a conventional microscope. The resulting light field microscope (LFM) can produce perspective flybys, focal stacks, and volume datasets from a single photograph, and therefore at a single instant in time. Finally, we've been working on multi-perspective imaging, currently with an eye towards visualizing urban landscapes, and animating light fields by interactively deforming its defining ray space. To help us make sense of all these uses of light fields, postdoc Hendrik Lensch and I co-taught a course recently on computational photography.
Graphics in the service of the humanities. My original training is in architecture (buildings, not computers), so I try to blend technology and the humanities in my research. One such effort is the Digital Michelangelo Project, a 5-year effort to create a three-dimensional digital archive of the statues of Michelangelo. Although the project is now dormant, we managed to create reasonably good computer models of 2 of the 10 statues we scanned. These models, and our raw scan data for the other 8, are available to scholars through our online archive. Another project at the juncture of technology and the humanities is our recent digitization of the 1,186 fragments of the Forma Urbis Romae, a giant marble map of ancient Rome. With guidance from Jennifer Trimble in Classics, we've created a geometric, photographic, and textual database of the map, and with help from Leo Guibas and his students, we're trying to solve the jigsaw puzzle algorithmically. So far, we've found about 20 matches. That may not sound like much, but it's more than human archaeologists have found in 30 years. (Here's a blurb on our first match, and here's an article from the Stanford Report describing the whole project.) Papers describing the project in more detail, and enumerating all the matches we've found, will appear soon in the Journal of Roman Archaeology and the Bullettino Della Commissione Archeologica Comunale di Roma. Another project with an archaeological flavor is the Cuneiform Tablet Visualization Project, in which we scanned, unwrapped, and non-photorealistically shaded the tablets' curved inscribed surfaces. Finally, the task of assembling archives of Michelangelo's statues and the Forma Urbis Romae has led Hector Garcia-Molina and I to think about the problems of creating digital archives of 3D artworks. Our efforts in this area have focused on real-time display of large models on low-cost PCs, efficient streaming of these models over networks of limited bandwidth, and protected viewing for non-licensed users via a remote rendering system.
Other fun follow-on projects. The ability to create high-resolution computer models of statues has presented us with some unexpected opportunities. For example, the Galleria dell'Accademia in Florence invited us to install a computer kiosk near Michelangelo's giant figure of David. Between November of 2002 and November of 2004, about 2 million visitors saw this kiosk. Our hope is that by allowing museum visitors to interactively rotate (and relight) our model of the statue, they can examine parts of the statue that are hard to see from the ground, like his head and hands. We've also begun making physical replicas of the David; I have one sitting in my display case, right next to the notorious Stanford bunny. Something else we're looking at is projecting images onto scanned objects using powerful (and carefully aligned) video projectors. Possible applications include visually canceling dirt as a planning aid for art conservators, recoloring ancient statues (that were originally painted), and, inspired by the Son et Lumiere shows of France, non-photorealistically coloring a statue to look like a 3D drawing or painting. We're beginning to build an array of 100 video projectors, so we'll have an opportunity to further explore these ideas.
Visualization and volume rendering. In a previous life, I worked on volume rendering. (We still maintain Phil Lacroute's popular Volpack package and an archive of volume datasets.) Although I am no longer actively working in this area, I've become interested in alternative ways of visualizing the three-dimensional structure of the natural world. Inspired by landmark books like Robert Hooke's Micrographia (1665) and Harold Edgerton's Stopping Time (1964), I have begun working on a book of volume renderings. The book will be called Volumegraphica. Another inactive project is my spreadsheet for images. Many people have asked me for this software; one day I might clean it up for distribution. Finally, I continue to think about the use of points as a display primitive. Although this approach proved impractical in 1985, the highly successful QSplat system is based on it, and the first Symposium on Point-Based Graphics was held in June 2004. I suspect we haven't heard the end of this story yet.
Click here for a list of
my publications.
Click here for a list of all the
publications from our laboratory.
Click here for a list of all the
research projects going on in our laboratory.
