Modeling background and segmenting moving objects from compressed video

Modeling background and segmenting moving objects are significant techniques for video surveillance and other video processing applications. Most existing methods of modeling background and segmenting moving objects mainly operate in the spatial domain at pixel level. In this paper, we present three new algorithms (running average, median, mixture of Gaussians) modeling background directly from compressed video, and a two-stage segmentation approach based on the proposed background models. The proposed methods utilize discrete cosine transform (DCT) coefficients (including ac coefficients) at block level to represent background, and adapt the background by updating DCT coefficients. The proposed segmentation approach can extract foreground objects with pixel accuracy through a two-stage process. First a new background subtraction technique in the DCT domain is exploited to identify the block regions fully or partially occupied by foreground objects, and then pixels from these foreground blocks are further classified in the spatial domain. The experimental results show the proposed background modeling algorithms can achieve comparable accuracy to their counterparts in the spatial domain, and the associated segmentation scheme can visually generate good segmentation results with efficient computation. For instance, the computational cost of the proposed median and MoG algorithms are only 40.4% and 20.6 % of their counterparts in the spatial domain for background construction.