Assignment 3: Camera Simulation

Due: Friday May 8th, 11:59PM

Please add a link to your final writeup on Assignment3Writeups.

Description

Most rendering systems generate images where the entire scene is in sharp focus, thus mimicking the effect of a pinhole camera. In contrast, real cameras contain multi-lens assemblies with different imaging characteristics such as limited depth of field, field distortion, vignetting and spatially varying exposure. In this assignment, you'll extend pbrt with support for a more realistic camera model that accurately simulates these effects. Specifically, we will provide you with descriptions of real wide-angle, normal and telephoto lenses, each composed of multiple lens elements. You will build a camera plugin for pbrt that simulates the traversal of light through these lens assemblies onto the film plane of a virtual camera. With this camera simulator, you'll explore the effects of focus, aperture and exposure. Once you have a working camera simulator, you will add simple auto-focus capabilities to your camera.

Step 1: Background Reading

Before beginning this assignment you should read the paper "A Realistic Camera Model for Computer Graphics" by Kolb, Mitchell, and Hanrahan. This paper is one of the assigned course readings. You should also review parts of Chapter 6 in pbrt.

Step 2: Getting Up and Running

Starter code and data files for Assignment 3 are located at http://graphics.stanford.edu/courses/cs348b-07/assignment3/assignment3.zip. In addition to source, this archive contains pbrt scene files you will render in this assignment, a collection of lens data files (.dat), and autofocus zone info files (.txt).

Modify pbrt

You'll need to make several modifications to pbrt before building the realistic camera plugin. First add the following virtual method to pbrt's Camera class in core/camera.h.

virtual void AutoFocus(Scene* scene) { }

Next, you'll need to call the AutoFocus method from Scene::Render in core/scene.cpp. Add the following line immediately following the calls to perform surface and volume integrator preprocessing.

camera->AutoFocus(this);

Lastly, you'll need to export the Sample object from pbrt's core shared library. Modify the definition of Sample in core/sampling.h to look like:

struct COREDLL Sample {

Building the camera simulator plugin

Your camera simulator will build into a separate pbrt plugin named realistic.so (or realistic.dll on Windows). We've included a Makefile for the myth machines as well as a Visual Studio project file. The Visual Studio project file assumes it is placed in the same directory as the other pbrt projects (win32/Projects) and that the project source files are placed in the cameras subdirectory of the pbrt source tree. If you wish to place the files elsewhere you will need to modify the project. Linux users can unzip the archive to any location, and the build process should work fine as long as PBRT_BASEPATH is set appropriately at the top of the Makefile. Note that the makefile copies the resulting shared object binary into your pbrt bin directory so no modifications need to be made to your environment's. LD_LIBRARY_PATH.

Browse the starter code

The majority of your implementation work in this assignment will be inside the RealisticCamera class defined in realistic.cpp. The other files provided in the archive are helper classes that will be useful to you when implementing autofocus, and are discussed in detail below.

Step 3: Setup the Camera

Notice that in this assignment the pbrt scenes specify that rendering should use the "realistic" camera plugin that you are implementing in this assignment. The realistic camera accepts a number of parameters defined in the pbrt scene file. The most important of these parameters include: the name of a lens data file, the distance between the film plane and the location of the back lens element (the one closest to the film), the diameter of the aperture stop, and the length of the film diagonal (distance from top left corner to bottom right corder of film). The values of these parameters are passed in to the constructor of the RealisticCamera class. All values in both the pbrt file and in the lens data file are in units of millimeters.

Camera "realistic" 
        "string specfile" "dgauss.50mm.dat" 
        "float filmdistance" 36.77
        "float aperture_diameter" 17.1
        "float filmdiag" 70 

The .dat files included with the starter code describe camera lenses using the format described in Figure 1 of the Kolb paper. Your RealisticCamera constructor should read the specified lens data file. In pbrt, the viewing direction is the positive z-direction in camera space. Therefore, your camera should be looking directly down the z-axis. The first lens element listed in the file (the lens element closest to the world, and farthest from the film plane) should be located at the origin in camera space with the rest of the lens system and film plane extending in the negative-z direction.

Each line in the file contains the following information about one spherical lens interface.

lens_radius  z-axis_intercept  index_of_refraction  aperture

More precisely:

• lens_radius: is the spherical radius of the element.

• z_axis_intercept: is thickness of the element. That is, it's the distance along the z-axis (in the negative direction) that seperates this element from the next.

• index of refraction: is the index of refraction on the camera side of the interface.

• aperture: is the aperture of the interface (rays that hit the interface farther than this distance from the origin don't make it through the lens element)

Note that exactly one of the lines in the data file will have lens_radius = 0. This is the aperture stop of the camera. It's maximum size is given by the aperture value on this line. It's actual size is specified as a parameter to the realistic camera via the pbrt scene file. Also note that the index of refraction of the world side of the first lens element is 1 (it's air).

Step 4: Generate Camera Rays

• You now need to implement the RealisticCamera::GenerateRay function. GenerateRay takes a sample position in image space (given by sample.imageX and sample.imageY) as an argument and should return a random ray from the camera out into the scene. To the rest of pbrt, your camera appears as any other camera. Given a sample position it returns a ray from the camera out into the world.

• Compute the position on the film plane the ray intersects from the values of sample.imageX and sample.imageY

• The color of a pixel in the image produced by pbrt is proportional to the irradiance incident on a film pixel (think of the film as a sensor in a digital camera). This value is an estimate of all light reaching this pixel from the world and through all paths through the lens. As stated in the paper, computing this estimate involves sampling this set of paths. The easiest way to sample all paths is to fire rays at the back element of the lens and trace them out of the camera by computing intersections and refractions at each lens interface (do not use the thick lens approximation from the paper to compute the direction of rays exiting the lens). Note that some of these rays will hit the aperture stop and terminate before exiting the front of the lens.

• GenerateRay returns a weight for the generated ray. The radiance incident along the ray from the scene is modulated by this weight before adding the contribution to the Film. You will need to compute the correct weight to ensure that the irradiance estimate produced by pbrt is unbiased. That is, the expected value of the estimate is the actual value of the irradiance integral. Note that the weight depends upon the sampling scheme used.

• Render each of the four scenes (hw3_dgauss.pbrt, hw3_wide.pbrt, hw3_fisheye.pbrt, hw3_telephoto.pbrt) using your realistic camera simulator. Example images rendered with 512 samples per pixel are given below: telephoto (top left), double gauss (top right), wide angle (bottom left) and fisheye (bottom right). Notice that the wide angle image is especially noisy -- why is that? Hint: look at the ray traces at the top of this web page.

4 samples per pixel

512 samples per pixel

Hints

1. ConcentricSampleDisk() is a useful function for converting two 1D uniform random samples into a uniform random sample on a disk. See p. 270 of the PBRT book.

2. You'll need some data structure to store the information about each lens interface as well as the aperture stop. For each lens interface, you'll need to decide how to test for intersection and how rays refract according to the change of index of refraction on either side (review Snell's law).
3. For rays that terminate at the aperture stop, return a ray with a weight of 0 -- pbrt tests for such a case and will terminate the ray instead of sending it out into the scene.
4. Pay attention to the coordinate system used to represent rays. Confusion between world space and camera space can be a major source of bugs.
5. As is often the case in rendering, your code won't produce correct images until everything is working just right. Try to think of ways that you can modularize your work and test as much of it as possible incrementally as you go. Use assertions liberally to try to verify that your code is doing what you think it should at each step. It may be worth your time to produce a visualization of the rays refracting through your lens system as a debugging aid (compare to those at the top of this web page).

Step 5: Play with Exposure and Depth of Field

• Render a second image of the scene using the double gauss lens with the aperture radius reduced by one half. What effects (name two) do you expect to see? Does your camera simulation produce this result? By decreasing the aperture radius by 1/2, how many stops have you decreased the resulting photographs exposure by?

Step 6: Autofocus

The autofocus mechanism in modern digital cameras sample light incident on subregions of the sensor (film) plane called autofocus zones (AF zones) and analyze these small regions of the image to compute focus. For example, an autofocus algorithm might look for the presence of high frequencies (sharp edges in the image) within an AF zone to signal the presence of an in focus image. You most likely have noticed the AF zones in the viewfinder of your own camera. As an example, the AF zones used by the autofocus system in the Nikon D200 are shown below.

In this part of the assignment, we will provide you a scene and a set of AF zones. You will need to use these zones to automatically determine the film depth for your camera so that the scene is in focus. Notice in the scene file autofocus_test.pbrt, the camera description contains an extra parameter af_zones. This parameter specifies the name of a text file that contains a list of AF zones. Each line in the file defines the bottom left and top right of a rectangular zone using 4 floating point numbers:

xleft xright ytop ybottom

These coordinates are relative to the top left corner of the film (the numbers will fall between 0.0 and 1.0). For example, a zone spanning the entire film plane would be given by 0.0 1.0 0.0 1.0. A zone spanning the top left quadrant of the film is 0.0 0.5 0.0 0.5.

Implementing Autofocus

You will now need to implement the AutoFocus method of the RealisticCamera class. In this method, the camera should modify it's film depth so that the scene is in focus.

There are many ways to go about implementing this part of the assignment. One approach is to shoot rays from within AF zones on the film plane out into the scene (essentially rendering a small part of the image), and then analyze the subimage to determine if that part of the image is in focus. The provided assignment 3 starter code is intended to help you implement autofocus in this manner. Here are some tips to get started with the provided code:

• The provided CameraSensor class is smilar to the pbrt Film class but does not write data to files and can be cleared using the Reset() method. You can use this class to produce subimages corresponding to each AF Zone. The CameraSensor::ComputeImageRGB() method will return a pointer to an array of floating point RGB values suitable for analysis.

• SimpleStratifiedSampler is a modified version of PBRT's Stratified sampler plugin that is adapted for the needs of this project. The sampler to be reset and reused multiple times. You will need to pass samples generated by this sampler to the Scene::Li function to obtain incoming radiance from the scene along this ray.

• We recommend using the "Sum-Modified Laplacian" operator described in Sree Nayar's "Shape From Focus" paper (http://graphics.stanford.edu/courses/cs348b-06/homework3/Nayar_CVPR92.pdf) as a sharpness heuristic.

To test your autofocusing algorithm, we provide three scenes that require the camera to focus using a single AF Zone. The images resulting from proper focusing within the given AF zone hw3_afdgauss_closeup.pbrt, hw3_afdgauss_bg.pbrt, and hw3_aftelephoto.pbrt are shown below. We show the location of the AF zone in each image as a white rectangle.

More autofocusing hints

• When generating subimages corresponding to the AF Zones, it will be important that you use enough samples per pixel to reduce noise that may make it difficult to determine the sharpness of the image. By experimentation, I've found that 256 total samples (16x16) using the Sum-Modified Laplacian gives stable results.
• Although the autofocusing approach described here involves analyzing the image formed within each AF zone, an alternative approach would be to compute focus by using the depth information of camera ray intersections with scene geometry. Using the thick lens approximation from the Kolb paper, you might be able to compute focus more efficiently than the approach described thus far.

• Note that the CameraSensor contains (commented out) code to dump the current image to disk. This can be very useful in debugging.

Step 7: Submission

• We've created wiki pages (FirstnameLastname/Assignment3) for all students in the class. Access to these pages is set up so that only you can view your page. Please compose your writeup on this page and link to it from the Assignment3Writeups page.

Grading

This assignment will be graded on a 4 point scale:

• 1 point:
• 2 points:
• 3 points:
• 4 points: