Assignment 6 - Image Matting and Composition

Due Date: Thursday March 1st, 11:59PM

Questions? Check out the Assignment 6 FAQ and discussion page.

In this assignment you will write code to separate a foreground object from a constant color background in an image and then place this foreground object in another scene of your choosing. In computer graphics terminology, you will extract (or lift) a matte from an image to isolate a foreground object, and then composite the foreground object with a new background using the matte. With such technology you can whisk your beloved TA off to far away lands and exotic beaches where he no longer has to grade homework assignments.

greenscreen imagematte newyork france

Download and build the starter code

1. Begin by downloading the Assignment 6 starter code and latest version of the libST source here. Libst has not changed in this assignment, but we are bundling it in with the assignment zip file just like in all the previous assignments.

2. Build the code. In this project, you will be modifying assignment6/matting.cpp. In the assignment6/ subdirectory we have placed a couple of image files. The files doodle.jpg, elephant.jpg, and kayvon.jpg are sample images of foreground objects in front of a green screen. We have also provided a number of backgrounds to test compositing the extracted foreground objects into. In this assignment you will also be given the opportunity to take your own pictures against a green screen.

Review of Premultiplied alpha

Recall from class the definition of pre-multiplied alpha. If a surface or image pixel has color C=(r,g,b) and opacity = a, we can either store the information as (r,g,b,a) or as (ra,ga,ba,a). In the later case, alpha has been premultiplied in with the component values for the color. This is useful for both performance and convenience, and is the representation I will use throughout this assignment. For example, the premultiplied representation of a 50% transparent white surface is (.5,.5,.5,.5).

The Problem

Given an image of an object in front of a green screen, you need to estimate the color and transparency of the foreground object at each pixel. As discussed in class, an image where the intensity of each pixel defined the opacity of a foreground object is called a matte. In this assignment you will extract (or lift, in film terminology) a matte from a photograph.

Unfortunately, given only a single image of the foreground object, it is impossible to analytically determine both the color and transparency of the foreground object even if the background color is known. For example, a brown pixel in the image might be the result of an opaque brown foreground surface, or it may be the result of a semi-transparent red surface in front of the green screen. The following is a summary of the mathematics of the problem.

Recall that the equation for a color that is the combination of a foreground and background colors is:

C = Co + (1-alpha)Ck

* 'C' is the color of the observed image pixel.
* 'Co' is the color of the foreground object with pre-multiplied alpha. 
* 'Ck' is the color of the background object 
* 'alpha' is the opacity of the foreground object (alpha=1 for an opaque object)

In the remainder of this page, I will refer to Ro, Go, and Bo as the red, green, and blue components of the foreground color (The same goes for R, G, B, and Rk, Gk, Bk). Note, these values include pre-multiplied alpha.

Matte extraction

Compositing

last edited 2007-02-28 06:42:45 by adsl-75-37-18-106