Double-Camera Fusion System for Animal-Position Awareness in Farming Pens

In livestock breeding, continuous and objective monitoring of animals is manually unfeasible due to the large scale of breeding and expensive labour. Computer vision technology can generate accurate and real-time individual animal or animal group information from video surveillance. However, the frequent occlusion between animals and changes in appearance features caused by varying lighting conditions makes single-camera systems less attractive. We propose a double-camera system and image registration algorithms to spatially fuse the information from different viewpoints to solve these issues. This paper presents a deformable learning-based registration framework, where the input image pairs are initially linearly pre-registered. Then, an unsupervised convolutional neural network is employed to fit the mapping from one view to another, using a large number of unlabelled samples for training. The learned parameters are then used in a semi-supervised network and fine-tuned with a small number of manually annotated landmarks. The actual pixel displacement error is introduced as a complement to an image similarity measure. The performance of the proposed fine-tuned method is evaluated on real farming datasets and demonstrates significant improvement in lowering the registration errors than commonly used feature-based and intensity-based methods. This approach also reduces the registration time of an unseen image pair to less than 0.5 s. The proposed method provides a high-quality reference processing step for improving subsequent tasks such as multi-object tracking and behaviour recognition of animals for further analysis.

Automated summary

In livestock breeding, continuous and objective monitoring of animals is challenging. Computer vision technology, specifically a double-camera system with image registration algorithms, can overcome the issues caused by occlusion and changes in appearance features. This approach uses a deformable learning-based registration framework with pre-registration and convolutional neural networks to improve registration accuracy. The proposed method shows significant improvement over commonly used feature-based and intensity-based methods and reduces registration time to less than 0.5 seconds.