| technologies | ||
| large camera arrays | ||
| large projector arrays | ||
| camera–projector arrays | ||
| optical effects | ||
| synthetic aperture photography | ||
| synthetic aperture illumination | ||
| synthetic confocal imaging | ||
| multi-camera vision systems | |
| omni-directional vision | |
| 1D camera arrays | |
| 2D camera arrays |
| 640 × 480 pixels × 30 fps × 128 cameras ÷ 18:1 MPEG = 512 Mbs |
|
| snapshot or video | |
| synchronized timing | |
| continuous streaming | |
| cheap sensors & optics | |
| flexible arrangement |
| How are the cameras arranged? | ||
| tightly packed high-performance imaging | ||
| widely spaced light fields | ||
| intermediate spacing synthetic aperture photography | ||
Cameras tightly
packed:
high-performance imaging
| high-resolution | ||
| by abutting the cameras’ fields of view | ||
| high speed | ||
| by staggering their triggering times | ||
| high dynamic range | ||
| mosaic of shutter speeds, apertures, density filters | ||
| high precision | ||
| averaging multiple images improves contrast | ||
| high depth of field | ||
| mosaic of differently focused lenses | ||
| Q. Can we align images this well? |
Cameras tightly
packed:
high-performance imaging
| high-resolution | ||
| by abutting the cameras’ fields of view | ||
| high speed | ||
| by staggering their triggering times | ||
| high dynamic range | ||
| mosaic of shutter speeds, apertures, density filters | ||
| high precision | ||
| averaging multiple images improves contrast | ||
| high depth of field | ||
| mosaic of differently focused lenses | ||
A virtual high-speed
video camera
[Wilburn, 2004 (submitted) ]
| 52 cameras, each 30 fps | |
| packed as closely as possible | |
| staggered firing, short exposure | |
| result is 1560 fps camera | |
| continuous streaming | |
| no triggering needed |
A virtual high-speed
video camera
[Wilburn, 2004 (submitted) ]
| 52 cameras, 30 fps, 640 × 480 | |
| packed as closely as possible | |
| short exposure, staggered firing | |
| result is 1536 fps camera | |
| continuous streaming | |
| no triggering needed | |
| scalable to more cameras | |
| limited by available photons | |
| overlap exposure times? |
Cameras tightly
packed:
high-X imaging
| high-resolution | ||
| by abutting the cameras’ fields of view | ||
| high speed | ||
| by staggering their triggering times | ||
| high dynamic range | ||
| mosaic of shutter speeds, apertures, density filters | ||
| high precision | ||
| averaging multiple images improves contrast | ||
| high depth of field | ||
| mosaic of differently focused lenses | ||
| overcomes one of photography’s key limitations | ||
| negative film = 250:1 (8 stops) | ||
| paper prints = 50:1 | ||
| [Debevec97] = 250,000:1 (18 stops) | ||
| hot topic at recent SIGGRAPHs | ||
Cameras tightly
packed:
high-X imaging
| high-resolution | ||
| by abutting the cameras’ fields of view | ||
| high speed | ||
| by staggering their triggering times | ||
| high dynamic range | ||
| mosaic of shutter speeds, apertures, density filters | ||
| high precision | ||
| averaging multiple images improves contrast | ||
| high depth of field | ||
| mosaic of differently focused lenses | ||
| scattering decreases contrast | |
| noise limits perception in low contrast images | |
| averaging multiple images decreases noise |
| scattering decreases contrast, but does not blur | |
| noise limits perception in low contrast images | |
| averaging multiple images decreases noise |
Cameras tightly
packed:
high-X imaging
| high-resolution | ||
| by abutting the cameras’ fields of view | ||
| high speed | ||
| by staggering their triggering times | ||
| high dynamic range | ||
| mosaic of shutter speeds, apertures, density filters | ||
| high precision | ||
| averaging multiple images improves contrast | ||
| high depth of field | ||
| mosaic of differently focused lenses | ||
| adjacent views use different focus settings | |
| for each pixel, select sharpest view |
Synthetic aperture photography
Synthetic aperture photography
Synthetic aperture photography
Synthetic aperture photography
Synthetic aperture photography
Synthetic aperture photography
Long-range
synthetic aperture photography
Synthetic aperture
photography
using an array of mirrors
| 11-megapixel camera | |
| 22 planar mirrors |
| technologies | ||
| array of projectors | ||
| array of microprojectors | ||
| single projector + array of mirrors | ||
| applications | ||
| bright display | ||
| autostereoscopic display [Matusik 2004] | ||
| confocal imaging [this paper] | ||
| works with any number of projectors ≥ 2 | |
| discrimination degrades if point to left of | |
| no discrimination for points to left of | |
| slow! | |
| poor light efficiency |
Synthetic
coded-aperture
confocal imaging
| different from coded aperture imaging in astronomy | |
| [Wilson, Confocal Microscopy by Aperture Correlation, 1996] |
Synthetic
coded-aperture
confocal imaging
Synthetic
coded-aperture
confocal imaging
Synthetic
coded-aperture
confocal imaging
Synthetic
coded-aperture
confocal imaging
Synthetic
coded-aperture
confocal imaging
Synthetic
coded-aperture
confocal imaging
Synthetic
coded-aperture
confocal imaging
Implementation using an array of mirrors
Synthetic aperture confocal imaging
Selective illumination using object-aligned mattes
Confocal imaging in scattering media
| small tank | ||
| too short for attenuation | ||
| lit by internal reflections | ||
Experiments in a large water tank
Experiments in a large water tank
| 4-foot viewing distance to target | |
| surfaces blackened to kill reflections | |
| titanium dioxide in filtered water | |
| transmissometer to measure turbidity |
Experiments in a large water tank
| stray light limits performance | |
| one projector suffices if no occluders |
| if vehicle is moving, no sweeping is needed! | |
| can triangulate from leading (or trailing) edge of stripe, getting range (depth) for free |
Strawman conclusions
on
imaging through a scattering medium
| shaping the illumination lets you see 2-3x further, but requires scanning or sweeping | |
| use a pattern that avoids illuminating the media along the line of sight | |
| contrast degrades with increasing total illumination, regardless of pattern |
Application
to
underwater exploration
| staff | ||
| Mark Horowitz | ||
| Marc Levoy | ||
| Bennett Wilburn | ||
| students | ||
| Billy Chen | ||
| Vaibhav Vaish | ||
| Katherine Chou | ||
| Monica Goyal | ||
| Neel Joshi | ||
| Hsiao-Heng Kelin Lee | ||
| Georg Petschnigg | ||
| Guillaume Poncin | ||
| Michael Smulski | ||
| Augusto Roman | ||
| collaborators | ||
| Mark Bolas | ||
| Ian McDowall | ||
| Guillermo Sapiro | ||
| funding | ||
| Intel | ||
| Sony | ||
| Interval Research | ||
| NSF | ||
| DARPA | ||
| (in reverse chronological order) | |||
| Spatiotemporal Sampling and Interpolation for Dense Camera Arrays | |||
| Bennett Wilburn, Neel Joshi, Katherine Chou, Marc Levoy, Mark Horowitz | |||
| ACM Transactions on Graphics (conditionally accepted) | |||
| Interactive Design of Multi-Perspective Images for Visualizing Urban Landscapes | |||
| Augusto Román, Gaurav Garg, Marc Levoy | |||
| Proc. IEEE Visualization 2004 | |||
| Synthetic aperture confocal imaging | |||
| Marc Levoy, Billy Chen, Vaibhav Vaish, Mark Horowitz, Ian McDowall, Mark Bolas | |||
| Proc. SIGGRAPH 2004 | |||
| High Speed Video Using a Dense Camera Array | |||
| Bennett Wilburn, Neel Joshi, Vaibhav Vaish, Marc Levoy, Mark Horowitz | |||
| Proc. CVPR 2004 | |||
| High Speed Video Using a Dense Camera Array | |||
| Bennett Wilburn, Neel Joshi, Vaibhav Vaish, Marc Levoy, Mark Horowitz | |||
| Proc. CVPR 2004 | |||
| The Light Field Video Camera | |||
| Bennett Wilburn, Michael Smulski, Hsiao-Heng Kelin Lee, and Mark Horowitz | |||
| Proc. Media Processors 2002, SPIE Electronic Imaging 2002 | |||