Improved visible to IR image transformation using synthetic data augmentation with cycle-consistent adversarial networks

Kyongsik Yun; Kevin Yu; Joseph Osborne; Sarah Eldin; Luan Nguyen; Alexander Huyen; Thomas Lu

doi:10.1117/12.2519121

13 May 2019 Improved visible to IR image transformation using synthetic data augmentation with cycle-consistent adversarial networks

Kyongsik Yun, Kevin Yu, Joseph Osborne, Sarah Eldin, Luan Nguyen, Alexander Huyen, Thomas Lu

Author Affiliations +

Proceedings Volume 10995, Pattern Recognition and Tracking XXX; 1099502 (2019) https://doi.org/10.1117/12.2519121
Event: SPIE Defense + Commercial Sensing, 2019, Baltimore, MD, United States

Abstract

Infrared (IR) images are essential to improve the visibility of dark or camouflaged objects. Object recognition and segmentation based on a neural network using IR images provide more accuracy and insight than color visible images. But the bottleneck is the amount of relevant IR images for training. It is difficult to collect real-world IR images for special purposes, including space exploration, military and fire-fighting applications. To solve this problem, we created color visible and IR images using a Unity-based 3D game editor. These synthetically generated color visible and IR images were used to train cycle consistent adversarial networks (CycleGAN) to convert visible images to IR images. CycleGAN has the advantage that it does not require precisely matching visible and IR pairs for transformation training. In this study, we discovered that additional synthetic data can help improve CycleGAN performance. Neural network training using real data (N = 20) performed more accurate transformations than training using real (N = 10) and synthetic (N = 10) data combinations. The result indicates that the synthetic data cannot exceed the quality of the real data. Neural network training using real (N = 10) and synthetic (N = 100) data combinations showed almost the same performance as training using real data (N = 20). At least 10 times more synthetic data than real data is required to achieve the same performance. In summary, CycleGAN is used with synthetic data to improve the IR image conversion performance of visible images.

Conference Presentation

Citation Download Citation

Kyongsik Yun, Kevin Yu, Joseph Osborne, Sarah Eldin, Luan Nguyen, Alexander Huyen, and Thomas Lu "Improved visible to IR image transformation using synthetic data augmentation with cycle-consistent adversarial networks ", Proc. SPIE 10995, Pattern Recognition and Tracking XXX, 1099502 (13 May 2019); https://doi.org/10.1117/12.2519121

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available