Assignment 4:
Portal-Masked Environment Map Sampling

Environment map

Description

Environment lighting allows us to render virtual scenes with complex lighting captured from the real world. An environment's illumination is captured in an environment map by storing the incoming radiance in a full sphere of directions around a specific point. An example environment map that you will be using in this assignment is shown above. As you can see, the spherical illumination is saved to a standard 2D image by mapping spherical coordinates to 2D texture coordinates. When rendering a scene, this mapping is inverted to build an infinitely large spherical light that surrounds the scene, where the light's radiance along a given direction corresponds to the captured radiance from the real world.

A major advantage of environment maps is computational efficiency: a detailed lighting setup can be captured in a single environment light, rather than with tens or hundreds of individual lights in a scene. This simplification does raise an important question: when sampling an environment light, which of the full sphere of possible directions should be sampled? The answer is importance sampling: directions that correspond to brighter parts of the environment map are sampled more frequently, as they will contribute more to the final image.

Simple importance sampling of the environment map works well for outdoor scenes, but consider an indoor scene where light can only enter through a few, relatively small, windows. In this situation, the brightest parts of the environment map will likely be occluded by walls or furniture, causing many samples of the environment map to be wasted and resulting in a noisy image. To solve this problem, you will be extending PBRT to support environment map "portals": square regions overlayed on windows in the scene that tell PBRT where to direct rays sampling the environment map.

Step 0: Setup

Like assignment 3, we will be providing starter code for this assignment. You will need to create a branch from the assignment4 branch in our starter code repository: github.com/stanford-gfx/cs348b-starter-code. These instructions assume you have already set up the starter_code remote on your local repo for assignment 3 (if not, simply follow the assignment 3 setup instructions first).

Run the following commands:

git fetch starter_code
git checkout -b assignment4 --track starter_code/assignment4
git push -u origin assignment4

You should now be in the assignment4 branch on your local repository with future commits set to go to your personal repository on GitHub.

Next, download the starter scene files for the assignment from here and extract them into the scenes/assignment4 directory.

Step 1: Background Reading

For this assignment, you will be implementing the algorithm described in Portal-Masked Environment Map Sampling from Bitterli et al. As preparation for the open ended final project, one of the goals of this assignment is to help you learn to independently read papers and implement the described algorithms. Therefore you will need to fully read and understand this paper, as the contents and details will not be repeated here; however, once you understand the algorithm, your implementation will require fewer lines of code than assignment 3.

You will be implementing the described algorithm as a new light in PBRT. Before proceeding, you should familiarize yourself with PBRT's implementation of a standard environment light (with no portal support): the InfiniteAreaLight class in src/lights/infinite.cpp. This code should provide a good introduction to PBRT's interface for lights, as well as some of the basics of environment maps and importance sampling. In addition to the code, refer to PBR section 12.6 for basic details about the infinite area light and environment mapping in general, and section 14.2.4 for details on the infinite area light's implementation of importance sampling.

Step 2: Rectifying the Environment Map

The starter code for this assignment is located in src/lights/portal.h and src/lights/portal.cpp. These files define the skeleton of a new PortalInfiniteLight class that you will be implementing. Structurally the class is based on PBRT's standard environment light: the InfiniteAreaLight; however, with the exception of basic functionality like reading the saved environment map, the algorithm described in the paper requires an entirely new implementation due to the new parameterization of the environment map and the more complex sampling strategy.

Parameter parsing functionality for PortalInfiniteLight is also provided in the starter code, which reads the standard InfiniteAreaLight parameters along with a point parameter P that defines the portal. The portal will always be a single rectangular quad with 4 points that define the corners in clockwise order. These portal corners are passed directly to the PortalInfiniteLight constructor for use in your code. To allow easy integration with existing scenes, the only difference between portal environment lights and regular environment lights in the .pbrt scene file is the presence of the P parameter. If the P parameter is not provided, the regular environment light is used: refer to scenes/assignment4/simple-portal.pbrt vs scenes/assignment4/simple-noportal.pbrt.

The algorithm for rectifying the environment map is described in section 2 of the paper. For this step, you should build and store the rectified environment map in the PortalInfiniteLight constructor and then implement PortalInfiniteLight::Le to support evaluating your rectified map along a given direction. The bulk of this step will be writing the code to transition between the rectified environment map's texture coordinates and world space directions. For now, don't worry about sampling the portal or implementing importance sampling - you will be testing on a scene with no geometry to simply confirm that your implementation correctly stores the environment map and that you have correctly implemented the transformations from world space directions to rectified coordinates.

Implementation Tips and Clarifications

  1. When resampling the spherical environment map into your rectified map, you need to choose the resolution of the rectified map and construct a new buffer for storing this rectified image. The exact resolution of the rectified map is up to you: the solution resamples into a square image with resolution in each dimension equal to the larger of the original environment map's dimensions.
  2. Use the sphericalLookup lambda function to get the value of the the spherical environment map along a direction. This function handles the WorldToLight transformation for you, so it expects a direction vector in world space.
  3. Be sure to distinguish between texture coordinates on the rectified environment map - uv coordinates ranging from 0 to 1 - and the paper's definition for rectified coordinates - $\alpha$ and $\beta$ - measured in radians and defining a hemisphere of directions. Both will likely exist as Point2f's in your code, but they cannot be used interchangeably - texture coordinates are needing for accessing the rectified environment map, while rectified coordinates can be transformed to 3D direction vectors using the mapping defined in the paper. You will need to determine the range of possible rectified coordinates and write a line or two of code to create a linear transformation between the two coordinate types.
  4. Your implementation can assume the portal is only accessed from one side. Specfically, you can assume that a ray launched from the origin of the scene towards your portal will hit the same side of the portal as all other rays in the scene.
  5. Although you will not be using the portal to sample rays in this section, rectified coordinates are defined relative to the portal's coordinate frame (shown in Figure 2 of the paper). Therefore you will need to process the passed in portal corner points in order to build a set of coordinate axes relative to the portal.
  6. Make sure the portal coordinate frame's z axis points in the correct direction as described in the paper. Depending on the order of the points passed into the constructor and your method for building a coordinate system, you may need to flip the z axis. Remember that your z axis should be correct relative to the rays that will be sampling the portal (described in the fourth tip).
  7. Try using a PBRT Transform to transform direction vectors between world space and the portal's coordinate frame.
  8. Build a set of dedicated helpers for transforming 2D rectified points to 3D direction vectors and vice versa.
  9. Use MIPMap's Lookup function when sampling the rectified map at a set of UV coordinates. The MIPMap class takes care of bilinear interpolation for you.

Once you have successfully implemented this section, you should be able to render scenes/assignment4/empty.pbrt to produce the following image of the environment map:

Empty

Step 3: Sampling the Rectified Map Through the Portal

Now that you have created the rectified environment map and can sample its value along a given direction, you need to implement support for importance sampling the environment map within the bounds of the portal. To get started, you should build an importance map of the rectified environment map in the PortalInfiniteLight constructor. The values that you should store in the importance map are described in the paper - you should also refer to InfiniteAreaLight's implementation and PBR 14.2.4 for how to build the importance map in PBRT's framework. Ultimately, you should set the distribution member variable provided in the starter code to a heap allocated SATDistribution2D that is initialized with the importance map.

Once you have finished building the importance map, you can implement Sample_Li and Pdf_Li. Refer to the beginning of PBR 12.2 for an overview of the requirements for these functions and how they should function. The algorithm you need to implement to sample the environment map with respect to the portal is described in section 3 of the paper; however, to simplify the assignment, the probability sampling and summed area table components of the paper have been implemented for you within a new SATDistribution2D class, which allows sampling values from a subset of the distribution. Specifically, the following functions, defined in src/lights/satdistribution.h, will provide the functionality you need:

Point2f SATDistribution2D::Sample(Point2f u, const Bounds2f &b, Float *pdf)
Float SATDistribution2D::Pdf(const Point2f &p, const Bounds2f &b)

These functions define similar interfaces to Distribution2D::SampleContinuous and Distribution2D::Pdf (described in PBR 13.6) with an additional Bounds2f parameter that restricts the distribution to a rectangular subset of its domain. At a high level, you should use this new functionality to sample the subset of the environment map visible through the portal from the current shading point.

While debugging this step, use the scenes/assignment4/simple-portal.pbrt and scenes/assignment4/simple-noportal.pbrt scenes to compare your implementation to the standard infinite area light. The reference solution's images are shown below at 16 spp:

portal portal

As you can see, your implementation of PortalInfiniteLight should be able to achieve a much cleaner image than InfiniteAreaLight at the same number of samples per pixel. Feel free to turn down the samples per pixel to speed up debugging: the quality difference should still be very noticeable even at 1 spp.

Implementation Tips and Clarification

  1. The SATDistribution2D is parameterized by uv coordinates from 0 to 1, not rectified coordinates. This means you will need to find the portal's projection onto the rectified map in terms of uv coordinates in order to correctly define the bounds parameter for sampling a subset of the distribution.
  2. Rays sampled directly from the BSDF may result in calls to PDF_Li with directions facing away from the portal - you should return 0 for these rays. Similarly, due to imprecision in the portal coordinates compared to the geometry of the wall, Sample_Li may be called with a point that lies very slightly behind the portal - you can return black / set the pdf to 0 in this case as well.
  3. Refer to InfiniteAreaLight for how to set the VisibilityTester parameter correctly in Sample_Li.
  4. When computing the Jacobian determinant, don't forget that the final equation in the paper is a derivative with respect to $\alpha$ and $\beta$, which are rectified coordinates. Your implementation will likely require the Jacobian determinant in terms of uv texture coordinates on the rectified map. Recall your linear transformation between these two types of coordinates from step 2: you should be able to apply the chain rule to compute $\frac{\partial(\overrightarrow{\omega)}}{\partial(u)\partial(v)}$. The end result should be a simple constant factor multiplied by the Jacobian determinant provided in the paper.
  5. If you notice minor fireflies in the bright region of your image (where the floor is lit by the sun) compared to the reference image above, you can try using nearest neighbor lookup of values in the environment map in Sample_Li (use the Lmap->Texel() function). This issue occurs due to a mismatch between the sampled PDF and the interpolated environment map value (note that this is a relatively insignificant issue - feel free to just use Lmap->Lookup() and ignore the fireflies).

Once you have successfully implemented this section, you should be able to render a slightly more complicated scene: scenes/assignment4/bathroom-portal.pbrt. Your result should look very close to the image shown below:

bathroom

For comparison, render scenes/assignment4/bathroom-noportal.pbrt to see the quality improvement from your implementation!

Step 4: Short Discussions

  • In step 2, we made the assumption that rays would only be launched towards one side of a given portal. How could you extend your implementation to support sampling rays from either side?
  • Explain why it is necessary to multiply by the inverse of the Jacobian when computing the probability of a given light sample.
  • Describe what occurs both visually and mechanically in the algorithm when the portal is larger or smaller than the actual window in the geometry.

Submission

Once you are done with the assignment, render simple-portal.pbrt and bathroom-portal.pbrt at 16 spp, then convert the results to png files using the following command:

 imgtool convert --tonemap [imagename].exr [imagename].png

Save the final pngs in scenes/assignment4/simple.png and scenes/assignment3/bathroom.png respectively.

Next, create a Submission.md file in the root of your repository that contains the following information:

  • Your answers to the short discussion questions in Step 4.
  • A description of any troubles you ran into while implementing the assignment, and how you overcame or worked around them.
  • Embedded copies of simple.png and bathroom.png using markdown syntax as described in the assignment 2 handout.

In total, your submission should include the following files:

  • All your changes to src/lights/portal.h, src/lights/portal.cpp (as well as any other changes you made to PBRT).
  • The images produced in step 3 as png files.
  • Your Submission.md file.

Ensure that all your changes have been committed and pushed to GitHub, then create a pull request from your assignment4 branch to your master branch. Add the TAs as reviews of the pull request and DO NOT merge your pull request.

Grading

  • 0 - Little or no work was done.
  • 1 - Significant effort was put into the assignment, but not everything works correctly, insufficient explanation given, or answers incorrect.
  • 2 - Everything mostly works in step 2 & 3. Specifically, your PortalInfiniteLight class needs to contain (not necessarily fully correct) implementations of the constructor, Le, Sample_Li, and Pdf_Li that follow the algorithm described in the paper: resampling the environment map into rectified coordinates and importance sampling the portal through the rectified map. Errors are explained, short answers sufficient, adequate discussion of troubles during implementation.
  • 3 - Everything works and your implementation produces observably less noisy results than PBRT's InfiniteAreaLight in the provided scenes (simple-portal.pbrt vs simple-noportal.pbrt and bathroom-portal.pbrt vs bathroom-noportal.pbrt). short answers are insightful and precise, detailed record of troubles during implementation.