Title: Two-Stream 3D Convolutional Neural Network for Skeleton-Based Action Recognition,
Abstract: It remains a challenge to efficiently extract spatialtemporal information
from skeleton sequences for 3D human action recognition. Although most recent
action recognition methods are based on Recurrent Neural Networks which present
outstanding performance, one of the shortcomings of these methods is the
tendency to overemphasize the temporal information. Since 3D convolutional
neural network(3D CNN) is a powerful tool to simultaneously learn features from
both spatial and temporal dimensions through capturing the correlations between
three dimensional signals, this paper proposes a novel two-stream model using
3D CNN. To our best knowledge, this is the first application of 3D CNN in
skeleton-based action recognition. Our method consists of three stages. First,
skeleton joints are mapped into a 3D coordinate space and then encoding the
spatial and temporal information, respectively. Second, 3D CNN models are
seperately adopted to extract deep features from two streams. Third, to enhance
the ability of deep features to capture global relationships, we extend every
stream into multitemporal version. Extensive experiments on the SmartHome
dataset and the large-scale NTU RGB-D dataset demonstrate that our method
outperforms most of RNN-based methods, which verify the complementary property
between spatial and temporal information and the robustness to noise.