Shape-and-behavior-encoded tracking of bee dances

Behavior analysis of social insects has garnered impetus in recent years and has led to some advances in fields like control systems and flight navigation. Manual labeling of insect motions required for analyzing the behaviors of insects requires significant investment of time and effort. In this paper, we propose certain general principles that help in simultaneous automatic tracking and behavior analysis, with applications in tracking bees and recognizing specific behaviors that they exhibit. The state space for tracking is defined using the position, orientation, and current behavior of the insect being tracked. The position and the orientation are parameterized using a shape model, whereas the behavior is explicitly modeled using a three-tier hierarchical motion model. The first tier (dynamics) models the local motions exhibited, and the models built in this tier act as a vocabulary for behavior modeling. The second tier is a Markov motion model built on top of the local motion vocabulary, which serves as the behavior model. The third tier of the hierarchy models the switching between behaviors, and this is also modeled as a Markov model. We address issues in learning the three-tier behavioral model, in discriminating between models, and in detecting and modeling abnormal behaviors. Another important aspect of this work is that it leads to joint tracking and behavior analysis instead of the traditional track-and-then-recognize approach. We apply these principles for tracking bees in a hive while they are executing the waggle dance and the round dance.