Broad Area Colloquium For AI-Geometry-Graphics-Robotics-Vision
(CS 528)

Finding Unusual Activity in Video

Jianbo Shi
University of Pennsylvania
Monday, January 26, 2004, 4:15PM
TCSeq 200


Imagine you are given a long video, possibly thousands of hours long, and you are asked to analyze the video to detect unusual events. This situation arises in applications such as video surveillance, health care monitoring, and biometric human identification.

Humans can extract essential characteristics of video events relatively quickly and easily. How to quantify this process in a computable form is an open problem. By definition, unusual events are rare, difficult to describe, and impossible to predict. At the first glance, this problem is ill defined. Any system that detects unusual events must sift through extremely large amount of minute statistical details to detect a few relevant bits.

Our research can be broadly summarized in two directions. In the first direction, we focus on the extraction of unusual video activities using a large set of simple image features. The work is concentrated on detecting a) unusual video segments of a fixed time length, and b) those of variable time length, varying potentially from a few seconds to a few days. Much of our work has been focused in this area, particularly on con-current unsupervised feature selection and data classification. Motivated by a similar problem in document-keyword analysis, we have developed a graph spectral based method for finding patterns in video events. In the second direction, we focus on development of image/video features, tools for operator feedback and large scale visualization. For specific application domains, it is important to introduce features that can detect more precisely the action event of interests. Ultimately such a monitoring system will be used by human operators. For this system to be useful, one must develop intuitive feedback and visualization tools for large video sets.

Experimentally, we have tested our algorithm on a variety of videos ranging from nursing home monitoring, poker game cheating, to roadway surveillance.

This is a joint work with Hua Zhong at CMU, and Mirko Visontai at U.Penn.

About the Speaker

Jianbo Shi is an assistant professor of Computer and Information Science at University of Pennsylvania. He studied Computer Science and Mathematics as an undergraduate at Cornell University where he received his B.A. in 1994. He received his Ph.D. degree in Computer Science from University of California at Berkeley in 1998.  From 1999 to 2002, he was a research faculty at Robotics Institute at Carnegie Mellon University.  His current research interests focus on image segmentation and recognition, human activity recognition in video. His other research interests include image/video retrieval, and vision based desktop computing.


Back to the Colloquium Page