High Speed Video Using a Dense Camera Array

 

Bennett Wilburn

Stanford University

Neel Joshi

Stanford University

Vaibhav Vaish

Stanford University

Marc Levoy

Stanford University

Mark Horowitz

Stanford University

 

Presented at CVPR 2004

 

Abstract

 

We demonstrate a system for capturing multi-thousand frame-per-second (fps) video using a dense array of cheap 30fps CMOS image sensors. A benefit of using a camera array to capture high speed video is that we can scale to higher speeds by simply adding more cameras. Even at extremely high frame rates, our array architecture supports continuous streaming to disk from all of the cameras. This allows us to record unpredictable events, in which nothing occurs before the event of interest that could be used to trigger the beginning of recording.

  Synthesizing one high speed video sequence using images from an array of cameras requires methods to calibrate and correct those cameras' varying radiometric and geometric properties. We assume that our scene is either relatively planar or is very far away from the camera and that the images can therefore be aligned using projective transforms. We analyze the errors from this assumption and present methods to make them less visually objectionable. We also present a method to automatically color match our sensors. Finally, we demonstrate how to compensate for spatial and temporal distortions caused by the electronic rolling shutter, a common feature of low-end CMOS sensors.

 

 

 

Figure 1: Our 52 camera high speed array. Figure 2: 1560fps image of a popping balloon.
 

Paper

Adobe Acrobat PDF (300KB)

 

Videos

fan_even.mpg
1560 fps video of a fan. Aligning rolling shutter images causes temporal shifts, which make the fan motion non-uniform.

fan_shifted.mpg
Changing the camera trigger times to offset the temporal shifts makes the fan motion uniform again. Rolling shutter spatial distortions are still evident in the misshapen fan blades.

fan_shifted_sliced.mpg
Rolling shutter images are diagonal planes in the spatiotemporal volume. Slicing along planes of constant time produces an undistorted video. The fan spins uniformly, and the blades have the correct shape.

balloon1_distorted.mpg
The first of two popping balloons in one sequence. We have not corrected for the rolling shutter distortion, so the balloon appears to pop from the bottom instead of from where we struck it.

balloon2_distorted.mpg
The second popping balloon, also uncorrected.

balloon2_sliced.mpg
After slicing, the balloon motion is shown correctly.

balloons.mpg
The final corrected video sequence showing both popping balloons.