21/2D visual servoing with respect to planar contours having complex and unknown shapes

In this paper we present a complete system for segmenting, matching, tracking, and visual servoing with respect to unknown planar contours. Our system can be used with arbitrary contours of any shape and without any prior knowledge of their models. The system is first shown the target view. A selected contour is automatically extracted and its image shape is stored. The robot and object are then moved and the system automatically identifies the target. The matching step is performed together with the estimation of the homography matrix between the two views of the contour. Then, a 2 1/2 D visual servoing technique is used to reposition the end-effector of a robot at the reference position relative to the planar contour The system has been successfully tested on several contours with very complex shapes such as leaves, keys, and coastal outlines of islands. Experiments using a ship mock-up without any artificial marker proves that the system can be applied to the ship-building industry.