An improved human-object interaction detection method based on short-term memory selection network

Chang Wang; Shiwei Ma

doi:10.1117/12.2579840

10 November 2020 An improved human-object interaction detection method based on short-term memory selection network

Chang Wang, Shiwei Ma

Proceedings Volume 11584, 2020 International Conference on Image, Video Processing and Artificial Intelligence; 1158417 (2020) https://doi.org/10.1117/12.2579840
Event: Third International Conference on Image, Video Processing and Artificial Intelligence, 2020, Shanghai, China

Abstract

Human-object interaction (HOI) detection task is defined as inferring all the < human, verb, object > triplets in the image, which helps computers to obtain a more comprehensive understanding of the visual scene. Most existing HOI detection methods focus on instance local features, and rarely consider the information from backgrounds. Our core idea is that the relationship between human, object and other backgrounds contains important cues to facilitate HOI detection. According to the short-term memory selection (STMS) mechanism, we regard the interaction relationship as the result of human and object stimulating the union area, and simulate the stimulation process by the recurrent neural network. The features in the union area of human and object are taken as the input of RNN, human and object are the two inputs of RNN, and the output is the representation of the interaction relationship. Combined with the visual features and spatial features of instances, a multi-stream network is utilized to detect HOIs in the image. Experiments on V-COCO and HICO-DET show that the proposed model achieves better performance, verifying the effectiveness of our method.

Citation Download Citation

Chang Wang and Shiwei Ma "An improved human-object interaction detection method based on short-term memory selection network", Proc. SPIE 11584, 2020 International Conference on Image, Video Processing and Artificial Intelligence, 1158417 (10 November 2020); https://doi.org/10.1117/12.2579840

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available