Temporal profile summarization and indexing for surveillance videos

Abstract

Surveillance videos are recorded continually and the retrieval of such videos currently still relies on human operators. Automatic retrieval has not reached a satisfactory accuracy. As an intermediate representation, this work develops multiple original temporal profiles of video to convey accurate temporal information in the video while keeping certain spatial characteristics. These are effective methods to visualizes surveillance video contents efficiently in a 2D temporal image, suitable for indexing and retrieving a large video database. We are aiming to provide a compact index that is intuitive and preserves most of the information in the video in order to avoid browsing extensive video clips frame by frame. By considering some of the properties of static surveillance videos, we aim at accentuating the temporal dimension in our visualization. We have introduced our framework as three unique methods that visualize different aspects of a surveillance video, plus an extension to non-static surveillance videos. In our first method "Localized Temporal Profile", by knowing that most surveillance videos are monitoring specific locations, we try to emphasize the other dimension, time, in our solution. we focus on describing all the events only in critical locations of the video. In our next method "Multi-Position Temporal Profile", we generate an all-inclusive profile that covers all the events in the video field of view. In our last method "Motion Temporal Profile" we perform in-depth analysis of scene motion and try to handle targets with non-uniform, non-translational motion in our temporal profile. We then further extend our framework by loosening the constraint that the video is static and including cameras with smooth panning motion as such videos are widely used in practice. By performing motion analysis on the camera, we stabilize the camera to create a panorama-like effect for the video, allowing us to utilize all of the aforementioned methods. The resulting profiles allows temporal indexing to each video frame, and contains all spatial information in a continuous manner. It also shows the actions and progress of events in the temporal profile. Flexible browsing and effective manipulation of videos can be achieved using the resulting video profiles.