Step Two - Color Segmentation with mixture of gaussians

The second step of the program uses a color segmentation algorithm with a mixture of gaussians that divides up the image based on the given training data. One set of training data is given for the ball, the other for the players. First let's take a look at what these layers look like for a given frame in the sequence:

Original Frame:

Ball Layer:

Player Layer (note that all values are either 0 or 1):

At first, one might say that the color segmentation has done a very poor job. After all, the ball is white, yet it looks like half the field has been selected in the process. At the same time, the players are dark colored, yet it appears that a siginificant portion of the bottom of the field has also been included. Could this be right?

The fact is that although the ball appears white to us, it really is made up of various shades of light green. When the ball is still, it appears to have more obvious difference from the terrain:

However, when the ball is in motion, the difference can be very minute. The color data for the ball must be inclusive enough to allow the ball to be tracked even when it is moving.

As you can see, the ball in motion is almost invisible to the eye when examined at the pixel level. Thus I had to make the filter extremely sensitive, so much so that it often picks up portions of the field in the process. Something similar happens with the player layer. Because some of the players are wearing white jerseys, those appear greenish at certain moments. Also some of the dark colors are really dark green at the pixel level. So the player layer picks up some of the darker parts of the field.

As a final demonstration, let me superimpose the original image with each of the two layers to demonstrate. As you can see, the ball layer includes the ball and usually seems to exclude the players. On the other hand, the player layer includes all of the players and has a little gap where the ball is located. In both, the ball has been highlited in red.

Superposition with the ball layer:

Superposition with the player layer:

So what now? We have our two layers, but it doesn't seem that we are much closer to locating the ball. I discuss the construction of what I call the "ball mask" in the next step.