Homework 1 (Due Oct-11-00)

Implement and experiment with "Lucas-Kanade".

If you use matlab do following steps:

• We provide an example image sequence on the leland cluster in: /usr/class/cs448a/DATA/SRI/
• You can read images from there with: t=17; im = imread(sprintf('1bw%d.png',t));
• Display it with: colormap(gray); imagesc(im);
• Now--- Implement Lucas-Kanade. As input it should take an area of interest (either use ginput and a fixed window size or specify a tracking mask with arbitraty shape with mask = double(roipoly);
• Track that area over the multiple frames. Use the matlab plot functions to visualize your results. Print it out and submit the prints as your homework.
• If your code doesn't work, debug it with artifical input images that you generate.
• If your code works on the example images, go and look for other sequences and try it on that as well.
• If you're stuck-- email us.

Step 0 (Initialization):
Label in the first image of your sequence (Image F) the region of interest (ROI). Either pick a center + some rectangular area with imagesc(im); xy = ginput(1) or choose an arbitrary ROI shape with imagesc(im); roi = double(roipoly);

In order to get all the indices of the region of interest, you can use inds = find(roi); . For example to sum over all Fx you can do sum(Fx(inds).^2) .

Fx(x,y) is the spatial gradient along the x-axis of image F at location x,y.

Fy(x,y) is the spatial gradient along the y-axis.

You can compute those gradients with the convolution we discussed in class. Pick a sigma (start with 3 for the first few iterations and then 1.5), and compute the convolution kernel. You can create that kernel by computing two vectors: gauss (the Gaussian function), and dgauss (the first derivative of the Gaussian function). Compute the outer product: kernel = gauss * dgauss' . Here's what I get if I do that for sigma = 2 and plot it in matlab with surf(kernel):

Convoluting an image with this kernel can be done using conv2() in matlab. You compute Fx with kernel=gauss * dgauss', and Fy with kernel=dgauss*gauss;

Ft (the temporal gradient of the image) can be computed in just subtracting G-F.

Step 3 (Re-warping). Rewarp G (or F). The rewarping should be done in sub-pixel accuracy. You can do that with interp2(... ,'cubic'); The XI and YI coordinates (as specified in matlab help) can be computed with: first use meshgrid() and then add to them the new U and V.

Step 4 If error to high, goto Step 1.