Mr. Collen Roller Profile

Collen Roller

at Air Force Research Lab

SPIE Involvement:

Author

Publications (2)

This will count as one of your downloads.

You will have access to both the presentation and article (if available).

DOWNLOAD NOW

This content is available for download via your institution's subscription. To access this item, please sign in to your personal account.

Email or Username Forgot your username?

Password Forgot your password?

Show

Keep me signed in

No SPIE account? Create an account

Proceedings Article | 16 June 2023 Poster + Paper

Skeleton-based human action recognition with a physics-augmented encoder-decoder network

Hongji Guo, Alexander Aved, Collen Roller, Erika Ardiles-Cruz, Qiang Ji

Proceedings Volume 12525, 125250K (2023) https://doi.org/10.1117/12.2664115

KEYWORDS: Action recognition, Convolution, Physics, Education and training, 3D modeling, Modeling, Visual process modeling, Neural networks, Deep learning

Read Abstract +

Human action recognition is important for many applications such as surveillance monitoring, safety, and healthcare. As 3D body skeletons can accurately characterize body actions and are robust to camera views, we propose a 3D skeleton-based human action method. Different from the existing skeleton-based methods that use only geometric features for action recognition, we propose a physics-augmented encoder and decoder model that produces physically plausible geometric features for human action recognition. Specifically, given the input skeleton sequence, the encoder performs a spatiotemporal graph convolution to produce spatiotemporal features for both predicting human actions and estimating the generalized positions and forces of body joints. The decoder, implemented as an ODE solver, takes the joint forces and solves the Euler-Lagrangian equation to reconstruct the skeletons in the next frame. By training the model to simultaneously minimize the action classification and the 3D skeleton reconstruction errors, the encoder is ensured to produce features that are consistent with both body skeletons and the underlying body dynamics as well as being discriminative. The physics-augmented spatiotemporal features are used for human action classification. We evaluate the proposed method on NTU-RGB+D, a large-scale dataset for skeleton-based action recognition. Compared with existing methods, our method achieves higher accuracy and better generalization ability.

Proceedings Article | 16 June 2023 Poster + Paper

Video-based complex human event recognition with a probabilistic transformer

Hongji Guo, Alexander Aved, Collen Roller, Erika Ardiles-Cruz, Qiang Ji

Proceedings Volume 12525, 125250J (2023) https://doi.org/10.1117/12.2664106

KEYWORDS: Transformers, Action recognition, Video, Data modeling, Performance modeling, Statistical modeling, Machine learning

Read Abstract +

Complex human events are high-level human activities that are composed of a set of interacting primitive human actions over time. Complex human event recognition is important for many applications, including security surveillance, healthcare, sports and games. Complex human event recognition requires recognizing not only the constituent primitive actions but also, more importantly, their long range spatiotemporal interactions. To meet this requirement, we propose to exploit the self-attention mechanism in the Transformer to model and capture the long-range interactions among primitive actions. We further extend the conventional Transformer to a probabilistic Transformer in order to quantify the event recognition confidence and to detect anomaly events. Specifically, given a sequence of human 3D skeletons, the proposed model first performs primitive action localization and recognition. The recognized primitive human actions and their features are then fed into the probabilistic Transformer for complex human event recognition. By using a probabilistic attention score, the probabilistic Transformer can not only recognize complex events but also quantify its prediction uncertainty. Using the prediction uncertainty, we further propose to detect anomaly events in an unsupervised manner. We evaluate the proposed probabilistic Transformer on FineDiving dataset and Olympics Sports dataset for both complex event recognition and abnormal event detection. The dataset consists of complex events composed of primitive diving actions. The experimental results demonstrate the effectiveness and superiority of our method against baseline methods.

My Library

You currently do not have any folders to save your paper to! Create a new folder below.

Folder Name

Folder Description

View contact details

UPDATE YOUR PROFILE

Is this your profile? Update it now.

Sign into your SPIE.org account

Don’t have a profile and want one?

Create an account on SPIE.org

Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks. You are receiving this notice because your organization may not have SPIE eBooks access.*

*Shibboleth/Open Athens users─please sign in to access your institution's subscriptions.

To obtain this item, you may purchase the complete book in print or electronic format on SPIE.org.

ORGANIZATIONAL
Sign in with credentials provided by your organization.

Organizational Username

Organizational Password

Show/Hide Password

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members:

Non-members: ADD TO CART

Keywords/Phrases

Search In:

Publication Years