Multi-dimension feature fusion for action recognition

Pei Dong; Jie Li; Junyu Dong; Lin Qi

doi:10.1117/12.2302485

10 April 2018 Multi-dimension feature fusion for action recognition

Pei Dong, Jie Li, Junyu Dong, Lin Qi

Proceedings Volume 10615, Ninth International Conference on Graphic and Image Processing (ICGIP 2017); 106151C (2018) https://doi.org/10.1117/12.2302485
Event: Ninth International Conference on Graphic and Image Processing, 2017, Qingdao, China

Abstract

Typical human actions last several seconds and exhibit characteristic spatio-temporal structure. The challenge for action recognition is to capture and fuse the multi-dimension information in video data. In order to take into account these characteristics simultaneously, we present a novel method that fuses multiple dimensional features, such as chromatic images, depth and optical flow fields. We built our model based on the multi-stream deep convolutional networks with the help of temporal segment networks and extract discriminative spatial and temporal features by fusing ConvNets towers multi-dimension, in which different feature weights are assigned in order to take full advantage of this multi-dimension information. Our architecture is trained and evaluated on the currently largest and most challenging benchmark NTU RGB-D dataset. The experiments demonstrate that the performance of our method outperforms the state-of-the-art methods.

Citation Download Citation

Pei Dong, Jie Li, Junyu Dong, and Lin Qi "Multi-dimension feature fusion for action recognition", Proc. SPIE 10615, Ninth International Conference on Graphic and Image Processing (ICGIP 2017), 106151C (10 April 2018); https://doi.org/10.1117/12.2302485

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available