Many of the targetted applications of the Stanford Multi-Camera Array (such as light field rendering, shape from light fields, synthetic aperture photography) require accurate calibration. Camera calibration gives us the geometric relation between the 3D coordinates of a point in the world and the 2D (pixel) coordinates of its image in a camera of the array. For the traditional pinhole camera model, calibration involves determining:

- lens distortion
*(2 parameters)* - focal length and principal point
*(4 intrinsic parameters)* - pose (position and viewing direction) of each camera, in a global
coordinate system
*(6 extrinsic parameters)*

This page describes our current implementation of camera calibration.

- Design Philosophy
- Feature Detection and Correspondence
- Nonlinear Optimization (Bundle Adjustment)
- Results
- TO DOs
- References

**Automation:**The procedure must be completely automated, requiring no manual intervention, especially clicking of feature correspondences. This is necessary due to the large number of cameras we have.**Accuracy:**The feature detection should be at sub-pixel accuracy, and there should be no mismatch in matching 3D points in the world to their locations in the (2D) image.**Generality:**We would like to be able to calibrate any arrangement of cameras our application requires, in any desired illumination. Our procedure simply requires the ability to take grayscale snapshots. It does not depend on colorimetric, radiometric or gamma correction, camera gains, etc. It can be the first stage of the acquisition pipeline if necessary.

- Edge detection
- Fitting lines to resulting edgels
- Finding intersections of line segments corresponding to square corners

Edge detection |
||

Line Fitting |
||

Final squares |

The nonlinear minization is a fairly
large scale problem: suppose we have *N* cameras, taking *S* synchronized snapshots.
Each camera is modeled by 12 parameters, in addition, there are *S-1* rigid motions of
the calibration grid. (The position of the grid in the first snapshot is used to define the
global coordinate system.) This leads to a *12N+6S-6* dimensional problem: these are the
number of parameters we are solving for. Suppose that on an average, we extract *P* point
correspondences from a snapshot, then we have a total of *D=P*N*S* observations. Typically,
we would have about *N=100* cameras, *S=15* snapshots, and about *P=120* point
correspondences per image. This is a 1284-dimensional search, with a 180,000 x 1284 jacobian.
In double precision, this would require about 1.7 gigabytes of storage, and take impractically
long to compute.

Fortunately, it is easy to show that the jacobian is very sparse. Each row of the jacobian corresponds to one ray through a point on the calibration grid and the center of one camera in the array. The only parameters this ray depends on are the 12 parameters or the camera, and the 6 representing the pose of the calibration grid. This means that each row of the jacobian can have at most 18 nonzero entries. Consequently, the effective width is a constant, independent of the number of cameras or snapshots. Exploiting this sparsity is necessary for a feasible implementation of the optimization.

RMS error | 0.3199 pixels |

Mean error | 0.2647 pixels |

Median error | 0.2342 pixels |

Standard Deviation | 0.1796 pixels |

Avg. Planarity Deviation | 0.9636 cm |

Camera centers, as computed by calibration, projected onto a photo of the array
itself. The projection was approximated by a homography computed by manually specifying
control points on the photograph to map the camera centers to.Control points (clicked manually) Superimposed control points and camera projections |

Histogram of reprojection errors

Cummulative distribution of reprojection errors

2D plot of reprojection errors, MATLAB .fig file

3D plot of camera centers, MATLAB .fig file

- Components of the calibration pipeline have to be better integrated.
- Port feature detector to Gandalf, for increased speed, better integration. Extend it to handle calibration objects of varying sizes.
- Finish code to better visualize computed camera geometry and reprojection errors.
- Build larger calibration objects (upto 1m x 1m) for wide area calibration. Check out Pacific Panels / Home Depot for large flat mounting surfaces.
- Port nonlinear minimization to C/C++, eliminating need for Matlab.
*(Great project for another CS 205 student!)*

- [1] A Flexible Technique
for Camera Calibration

Zhenghyou Zhang, MSR Tech. Report MSR-TR-98-71 - [2] The Camera
Calibration Toolbox (Matlab)

Jean-Yves Bouget. - [3] Intel OpenCV Computer Vision Library (C++)
- [4] Robust Multi-Camera Calibration

Vaibhav Vaish (CS 205 Winter 2002-3 class project)

Last update: May 7th, 2003.