Automatic moving object extraction toward content-based video representation and indexing

In this paper, we present a novel semantic video object generation and temporal tracking technique for providing content-based video representation and indexing. In our system, the homogeneous image regions with accurate boundaries are first obtained by integrating the results of color edge detection and similarity-based region growing procedures. Then the semantic objects of meaning to human users are generated by an automatic seeded region aggregation or human interaction procedure. These generated semantic objects are then tracked along the time axis for exploiting their temporal correspondences among frames. The video sources in databases can be accessed through these individual video objects obtained and they are represented by a set of visual features, such as its trajectory, set of boundary pixels, area of the object region, shape, color histogram, orientation, and regularity. A learning-based optimization procedure is used for selecting the suitable dimensional weighting coefficients, so that the feature-based object similarity corresponds to concept-based similarity directly. Moreover, a seeded semantic video content clustering technique for providing cluster-based hierarchical video indexing, retrieval, and browsing is proposed. (C) 2001 Academic Press.