Humans can extract essential characteristics of video events relatively quickly and easily. How to quantify this process in a computable form is an open problem. By definition, unusual events are rare, difficult to describe, and impossible to predict. At the first glance, this problem is ill defined. Any system that detects unusual events must sift through extremely large amount of minute statistical details to detect a few relevant bits.

Our research can be broadly summarized in two directions. In the first direction, we focus on the extraction of unusual video activities using a large set of simple image features. The work is concentrated on detecting a) unusual video segments of a fixed time length, and b) those of variable time length, varying potentially from a few seconds to a few days. Much of our work has been focused in this area, particularly on con-current unsupervised feature selection and data classification. Motivated by a similar problem in document-keyword analysis, we have developed a graph spectral based method for finding patterns in video events. In the second direction, we focus on development of image/video features, tools for operator feedback and large scale visualization. For specific application domains, it is important to introduce features that can detect more precisely the action event of interests. Ultimately such a monitoring system will be used by human operators. For this system to be useful, one must develop intuitive feedback and visualization tools for large video sets.

Experimentally, we have tested our algorithm on a variety of videos ranging from nursing home monitoring, poker game cheating, to roadway surveillance.

This is a joint work with Hua Zhong at CMU, and Mirko Visontai at U.Penn.

Back to the Colloquium Page