Cross-View Action Recognition Based on a Statistical Translation Framework

Actions captured under view changes pose serious challenges to modern action recognition methods. In this paper, we propose an effective approach for cross-view action recognition based on a statistical translation framework, which boils down to estimation of visual word transfer probabilities across views. Specifically, local features are extracted from action video frames and form bags of words based on k-means clustering. Though the appearance of an action may vary due to view changes, the underlying transfer tendency between visual words across views can be exploited. We propose two methods to measure the visual-word-based transfer relationship that are eventually based on frequency counts of word pairs. In the first method, word transfer probabilities are estimated by maximizing the likelihood of a shared action set with the EM algorithm. In the second method, word transfer probabilities are estimated by using likelihood-ratio tests. The two methods achieve comparable results and perform better when they are combined. For cross-view action classification, we compute action transfer probabilities based on the estimated word transfer probabilities and then implement a K-NN-like classification based on action video transfer probabilities. We verified our method on the public multiview IXMAS dataset and the WVU dataset. Promising results are obtained compared with state-of-the-art methods.