Camera 2.0: New computing platforms for computational photography

This cell phone... ...took this picture. Version 2.0 of the Frankencamera, our experimental open-source camera platform.
Can your point-and-shoot camera do better?
Here are more N95 pictures of Stanford. And here is another N95 album. 		The camera runs Linux, and its metering, focusing, demosaicing, denoising, white balancing, and other post-processing algorithms are programmable. The current version takes Canon EOS lenses.

Overview

Computational photography refers broadly to sensing strategies and algorithmic techniques that enhance or extend the capabilities of digital photography. The output of these techniques is an ordinary photograph, but one that could not have been taken by a traditional camera. Representative techniques include high dynamic range imaging, flash-noflash imaging, coded aperture and coded exposure imaging, photography under structured illumination, multi-perspective and panoramic stitching, digital photomontage, all-focus imaging, and light field imaging.

Although interest in computational photography has steadily increased among graphics and vision researchers, progress in some aspects of this area has been hampered by the lack of a portable, programmable camera platform with enough image quality and computing power to be used for everyday photography. To address this problem, we are pursuing two subprojects:

  1. Computational Photography on Cell Phones. Over the past five years, the cameras in cell phones have improved dramatically in resolution, optical quality, and photographic functionality. (See 1st and 2nd images above.) Moreover, camera phones offer features that dedicated cameras do not - wireless connectivity, a high-resolution display, 3D graphics, and high-quality audio. Finally and perhaps most importantly, these platforms run real operating systems, which vendors have begun opening to third-party developers. We are taking advantage of these trends to develop computational photography applications for commerically available cell phones.

  2. The Stanford Frankencamera. Despite these encouraging trends, there are computational photography experiments that simply cannot be implemented on today's cell phones. either because the cameras' sensor or optics aren't good enough, the computing resources aren't powerful enough, or the APIs connecting the camera to the computing are too restrictive. We are therefore building an open-source camera platform that runs Linux, is fully programmable (including its digital signal processor) and connected to the Internet, and accommodates SLR lenses and SLR-quality sensors. Our current prototype (3rd and 4th images above) is constructed from off-the-shelf parts, in some cases borrowed from dead cameras. It's also ugly - hence the name. Our goal is to distribute this platform at minimal cost to computational photography researchers and courses worldwide.
Camera 2.0, which began as a collaborative project between the Stanford Computer Graphics Laboratory and the Nokia Research Center Palo Alto Laboratory, is currently also supported by Adobe Systems, Kodak, Hewlett-Packard, and the Walt Disney Company.

People

Recent papers in this area:

Gaussian KD-Trees for Fast High-Dimensional Filtering
Andrew Adams Natasha Gelfand, Jennifer Dolson, Marc Levoy
Proc. SIGGRAPH 2009
Spatially Adaptive Photographic Flash
Rolf Adelsberger, Remo Ziegler, Marc Levoy, Markus Gross
Technical Report 612, ETH Zurich, Institute of Visual Computing, December 2008.
Viewfinder Alignment
Andrew Adams, Natasha Gelfand, Kari Pulli
Proc. Eurographics 2008

Publicity about the project

We'd love to hear from you, but first check out our list of frequently asked questions.

Left: Marc Levoy. Right: Andrew Adams with camera.
Behind them is the Stanford Multi-Camera Array.
Image by Linda Cicero.

Other links

© 2007-2009 Marc Levoy
Last update: November 13, 2009 02:28:42 PM