Efficient visual information indexation for supporting actionable intelligence and knowledge generation

Desale F. Nurye; Yagmur Aktas; Jorge García

doi:10.1117/12.2680257

17 October 2023 Efficient visual information indexation for supporting actionable intelligence and knowledge generation

Desale F. Nurye, Yagmur Aktas, Jorge García

Proceedings Volume 12742, Artificial Intelligence for Security and Defence Applications; 1274207 (2023) https://doi.org/10.1117/12.2680257
Event: SPIE Security + Defence, 2023, Amsterdam, Netherlands

Abstract

Visual indexing, or the ability to search and analyze visual media such as images and videos, is important for law enforcement agencies because it can speed up criminal investigations. As more and more visual media is created and shared online, the ability to effectively search and analyze this data becomes increasingly important for investigators to do their job effectively. The major challenges for video captioning include accurately recognizing the objects and activities in the image, understanding their relationships and context, generating natural and descriptive language, and ensuring the captions are relevant and useful. Near real-time processing is also required in order to facilitate agile forensic decision making and prompt triage, hand-over and reduction of the amount of data to be processed by investigators or subsequent processing tools. This paper presents a captioning-driven efficient video analytic which is able to extract accurate descriptions of images and videos files. The proposed approach includes a temporal segmentation technique providing the most relevant frames. Subsequently, an image captioning approach has been specialized to describe visual media related to counter-terrorism and cybercrime for each relevant frame. Our proposed method achieves high consistency and correlation with human summary on SumMe dataset, outperforming previous similar methods.

Conference Presentation

(2023) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Desale F. Nurye, Yagmur Aktas, and Jorge García "Efficient visual information indexation for supporting actionable intelligence and knowledge generation", Proc. SPIE 12742, Artificial Intelligence for Security and Defence Applications, 1274207 (17 October 2023); https://doi.org/10.1117/12.2680257

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available