Assignment 5 - High Dynamic Range Photography
Due Date: Thursday February 22nd, 11:59PM
Questions? Check out the Assignment 5 FAQ and discussion page.
In this assignment you'll explore high dynamic range photography. You'll take your own photos, generate a high dynamic range image from them, and explore a couple of ways of displaying HDR images on low dynamic range displays.
Steps
Download and build the starter code
Begin by downloading the Assigment 5 starter code and latest version of the libST source here. Note that you must use the new version of libST, it has been updated since the previous assignment.
- Build libST. Building libST should require the same steps on your development system as it did in previous assignments. A number of changes have been made to the code which are significant for this assignment. See the descriptions below for the full details of the changes, which you should look over and understand at a high level.
Build the assignment 5 starter code. The subdirectory /assignment5 contains the starting code for your project. This directory should contain the C++ source file hdr.cpp. You will provide us your modified version of hdr.cpp as part of this assignment's handin. The directory also response.h, response.cpp, and some support code in the directory /tnt. You should understand the interface provided in response.h and response.cpp since you will use it in your code. You are not responsible for any of the /tnt code. Finally, we've provided d40.cr, a response curve file for the Nikon D40. If you use our cameras this file will work, but if you have your own camera you must generate your own camera response curve. See the instructions below for how to do that.
4. If you want to start working on the assignment before you get to take your own photos, download the sample photos and hdr images from here. Note that part of the assignment, and your grade, is taking your own photos. We are only providing these so you can get started sooner.
Understanding Changes to libST
There have been a few changes to libST, but you are already familiar with much of the code.
- STColor - This class has been updated with many overloaded operators and a couple of helper functions. Be familiar with what's available here. This is the representation we use for pixels in an HDR image, so most of your code will deal with this class.
STHDRImage - This class is analogous to the STImage class and has almost equivalent functionality. The main difference is that each pixel is an STColor. STColors are floating point and can take on any valid float value. However, when displayed, they will be clamped to [0, 1]. We've added an operator()(int x, int y) so accessing pixels uses the simple notation hdrimage(x,y) (which returns an STColor&. Note the same operator has been implemented for STImage.
Review of Camera Imaging and HDR
The input to a camera is the radiance of the scene. Light enters the camera, passes through the lens system and aperature producing some irradiance Ei at each pixel i. The camera can control how much energy actually hits the sensors by increasing/decreasing the shutter speed. By integrating the irradiance Ei over this shutter time dt we get the sensor exposure, Xi. The many steps from this point to producing an actual image (film exposure, development, digital conversion, remapping) act as a non-linear function which maps these input exposures in the range [0,infinity], to image values Zi in the range [0,255]. This function f(Xi) = Zi is called the camera response curve.
Our goal is to use some output images to get the input Xi values. Therefore, we actually want to consider the inverse function f-1(Zi) = Xi, and by substitution f-1(Zi) = Ei*dt. Taking the natural logarithm we have ln(f^-1(Zi)) = ln(Ei) + ln(dt). We'll rename this function g(Zi) = ln(Ei) + ln(dt). This is the representation we'll actually use for the response curve (given a pixel value, what is the ln(exposure) ). Note that this represents exactly the same information, we've just adjusted the representation a bit. This is convenient for our purposes because different shutter speeds are equivalent to translations of this function.
Here we show a sample response curve. XXX Image
Answer These Questions
(Please include answers to the following with your handin)
Taking Photographs
Two D40's, available 2 hours at a time, signup
What to take pictures of, how to use the camera, # photos, range of shutter speeds
Creating an HDR Image
An HDR image is just the irradiance values Ei at each pixel due to the scene. In other words, an HDR image represents the actual power incident on each sensor, before any of the other effects of the camera after the lens and aperature, have taken place. Assume we have the g(Zi) representation of the response curve and just a single input image with known shutter speed dt. Then we could simply rearrange the response curve formula to find Ei:
g(Zi) = ln(Ei) + ln(dt)
ln(Ei) = g(Zi) - ln(dt)
Ei = exp( g(Zi) - ln(dt) )
So why can't we just use a single photograph? The response curve can't give pixel values less than 0 or greater than 255, so all exposures above a certain level will result in a response of 255 and similarly for exposures below a certain level. This means that multiple exposure values map to the same response pixel value, so the function cannot actually be inverted. Therefore the above equations cannot determine the actual exposure value if the input pixel value is too small or too large.
So we need enough photographs such that every pixel is within a usable range in at least one photo (best if its, e.g., in [64,192]). Using this method, j photos will result in j exposure estimates for each pixel (that is, we have Eij = exp( g(Zij) - ln(dt_j) ) for each pixel i in each image j). Then we need to combine them somehow. To do this we'll weight them so that Eij will have high weight if pixel Zij has a high confidence value, i.e. those in the middle of the response curve. We've already created a weight function for you (CameraResponse::Weight()) which accomplishes this. Then we just calculate a weighted average (which we do on the ln values, not the actual values):
ln (Ei) = sum_over_j( weight(Zij)*(g(Zij) - ln(dt_j)) ) / sum_over_j( weight(Zij) )
and the Ei can easily be found.
You should implement this function in recover_hdr(). Only the arguments to the function are necessary to accomplish this. Carefully read the comments for the functions of the CameraResponse class to make sure you're getting the values you expect (i.e. whether you're getting the value itself or ln(value)).
Taking a Virtual Photograph
Displaying an HDR Image: Linear Mapping
Displaying an HDR Image: Tone Mapping
Hints for Getting Started
Grading
- 1 stars -
- 2 stars -
- 3 stars -
- 4 stars -
Submission Instructions
We would like submission to be in the form of a single .zip archive. This archive should contain your modified version of hdr.cpp, a text file containing answers to assignment questions, and your photo stack.
Please email this zip file to cs148-win0607-staff@lists.stanford.edu before the deadline. Please make the subject line of this email "CS148 Assignment 5 Handin".