Combined Top-Down/Bottom-Up Segmentation

We construct an image segmentation scheme that combines top-down (TD) with bottom-up (BU) processing. In the proposed scheme, segmentation and recognition are intertwined rather than proceeding in a serial manner. The TD part applies stored knowledge about object shapes acquired through learning, whereas the BU part creates a hierarchy of segmented regions based on uniformity criteria. Beginning with unsegmented training examples of class and nonclass images, the algorithm constructs a bank of class-specific fragments and determines their figure-ground segmentation. This fragment bank is then used to segment novel images in a TD manner: The stored fragments are first used to recognize images containing class objects and then to create a complete cover that best approximates these objects. The resulting TD segmentation is then integrated with BU multiscale grouping to better delineate the object boundaries. Our experiments, applied to a large set of four classes (horses. pedestrians, cars, and faces), demonstrate segmentation results that surpass those achieved by previous TD or BU schemes. The main novel aspects of this work are the fragment learning phase, which efficiently learns the figure-ground labeling of segmentation fragments, even in training sets with high object and background variability, combining the resulting TD segmentation with BU criteria, and the use of segmentation to improve recognition.