CNN-fusion architecture with visual and thermographic images for object detection

Ndidiamaka Adiuku; Nicolas P. Avdelidis; Gilbert Tang; Angelos Plastropoulos; Suresh Perinpanayagam

doi:10.1117/12.2665984

12 June 2023 CNN-fusion architecture with visual and thermographic images for object detection

Ndidiamaka Adiuku, Nicolas P. Avdelidis, Gilbert Tang, Angelos Plastropoulos, Suresh Perinpanayagam

Proceedings Volume 12536, Thermosense: Thermal Infrared Applications XLV; 125360S (2023) https://doi.org/10.1117/12.2665984
Event: SPIE Defense + Commercial Sensing, 2023, Orlando, Florida, United States

Abstract

Mobile robots performing aircraft visual inspection play a vital role in the future automated aircraft maintenance, repair and overhaul (MRO) operations. Autonomous navigation requires understanding the surroundings to automate and enhance the visual inspection process. The current state of neural network (NN) based obstacle detection and collision avoidance techniques are suitable for well-structured objects. However, their ability to distinguish between solid obstacles and low-density moving objects is limited, and their performance degrades in low-light scenarios. Thermal images can be used to complement the low-light visual image limitations in many applications, including inspections. This work proposes a Convolutional Neural Network (CNN) fusion architecture that enables the adaptive fusion of visual and thermographic images. The aim is to enhance autonomous robotic systems’ perception and collision avoidance in dynamic environments. The model has been tested with RGB and thermographic images acquired in Cranfield’s University hangar, which hosts a Boeing 737-400 and TUI hangar. The experimental results prove that the fusion-based CNN framework increases object detection accuracy compared to conventional models.

Conference Presentation

Citation Download Citation

Ndidiamaka Adiuku, Nicolas P. Avdelidis, Gilbert Tang, Angelos Plastropoulos, and Suresh Perinpanayagam "CNN-fusion architecture with visual and thermographic images for object detection", Proc. SPIE 12536, Thermosense: Thermal Infrared Applications XLV, 125360S (12 June 2023); https://doi.org/10.1117/12.2665984

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available