Traffic police command gesture recognition technology based on machine vision and two-stream spatio-temporal attention graph convolutional network

Yuan Li

doi:10.1117/12.2684174

1 August 2023 Traffic police command gesture recognition technology based on machine vision and two-stream spatio-temporal attention graph convolutional network

Yuan Li

Author Affiliations +

Proceedings Volume 12754, Third International Conference on Computer Vision and Pattern Analysis (ICCPA 2023); 127543J (2023) https://doi.org/10.1117/12.2684174
Event: 2023 3rd International Conference on Computer Vision and Pattern Analysis (ICCPA 2023), 2023, Hangzhou, China

Abstract

For the requirement of automatic recognition of traffic police gestures in complex backgrounds based on vision sensors for driverless cars, we propose a method for traffic police gesture action recognition based on two-stream spatio-temporal attention graph convolutional network (2s-AGCN) with two different dimensional skeletal data. Firstly, detect the commanding traffic policeman in the video, extract the 2D and 3D skeletal data with the pose estimation algorithm to reduce the influence of complex background and joint overlap on action recognition, then, build the spatio-temporal graph model ; After that, we construct a 2s-AGCN network, input 2D and 3D skeletal sequences into the network to learn the spatio-temporal features of gesture actions. Finally, a fusion of the two-stream information is done and then output the final traffic police gesture category. 2s-AGCN uses Non-Local and TopK at the spatial level to focus on all nodes directly, selecting the strongest K neighbors of interaction strength; Temporal attention is used to focus on the frames that have higher contribution. The ablation study is done on the dataset CTPGD, and the results show that the method significantly improves the recognition accuracy of traffic police command gesture actions, especially those with overlapping skeleton points.

Citation Download Citation

Yuan Li "Traffic police command gesture recognition technology based on machine vision and two-stream spatio-temporal attention graph convolutional network", Proc. SPIE 12754, Third International Conference on Computer Vision and Pattern Analysis (ICCPA 2023), 127543J (1 August 2023); https://doi.org/10.1117/12.2684174

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
9 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Police

Gesture recognition

Action recognition

Feature fusion

Data fusion

Feature extraction

Video

Show All Keywords

Keywords/Phrases

Search In:

Publication Years