Sat U-net: a fusion based method for forensic splicing localization in satellite images

János Horváth; Ziyue Xiang; Edoardo Daniele Cannas; Paolo Bestagini; Stefano Tubaro; Edward J. Delp III

doi:10.1117/12.2616150

27 May 2022 Sat U-net: a fusion based method for forensic splicing localization in satellite images

János Horváth, Ziyue Xiang, Edoardo Daniele Cannas, Paolo Bestagini, Stefano Tubaro, Edward J. Delp III

Proceedings Volume 12100, Multimodal Image Exploitation and Learning 2022; 1210002 (2022) https://doi.org/10.1117/12.2616150
Event: SPIE Defense + Commercial Sensing, 2022, Orlando, Florida, United States

Abstract

Satellite images are widely available to the public. These satellite images are used in various fields including natural disaster analysis, meteorology and agriculture. As with any type of images, satellite images can be altered using image manipulation tools. A common manipulation is splicing, i.e., pasting on top of an image a region coming from a different source image. Most manipulation detection methods designed for images captured by “consumer cameras" tend to fail when used with satellite images. In this paper we propose a machine learning approach, Sat U-Net, to fuse the results of two exiting forensic splicing localization methods to increase their overall accuracy and robustness. Sat U-Net is a U-Net based architecture exploiting several Transformers to enhance the performance. Sat U-Net fuses the outputs of two unsupervised splicing detection methods, Gated PixelCNN Ensemble and Vision Transformer, to produce a heatmap highlighting the manipulated image region. We show that our fusion approach trained on images from one satellite can be lightly retrained on few images from another satellite to detect spliced regions. We compare our approach to well-known splicing detection methods (i.e., Noiseprint) and segmentation techniques (i.e., U-Net and Nested Attention U-Net). We conducted our experiments on two large datasets: one dataset contains images from Sentinel 2 satellites and the other one contains images from Worldview 3 satellite. Our experiments show that our proposed fusion method performs well when compared to other techniques in localizing spliced areas using Jaccard Index and Dice Score as metrics on both datasets.

Citation Download Citation

János Horváth, Ziyue Xiang, Edoardo Daniele Cannas, Paolo Bestagini, Stefano Tubaro, and Edward J. Delp III "Sat U-net: a fusion based method for forensic splicing localization in satellite images", Proc. SPIE 12100, Multimodal Image Exploitation and Learning 2022, 1210002 (27 May 2022); https://doi.org/10.1117/12.2616150

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
18 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Image fusion

Satellites

Satellite imaging

Earth observing sensors

Transformers

Fusion splicing

Forensic science

Show All Keywords

Keywords/Phrases

Search In:

Publication Years