20 March 2024 Improved SmapGAN remote sensing image map generation based on multi-head self-attention and carafe
Zhipeng Ding, Ben Wang, Shuifa Sun, Yongheng Tang, Ren Zhuang, Wenbo Liu
Author Affiliations +
Abstract

The changes in ground roads, buildings, and occurrences of natural disasters lead to mismatches between the actual ground conditions and existing maps. Through style transfer between real-time remote sensing images and maps, map content can be rapidly generated and updated. However, in existing methods for generating maps from remote sensing images based on SmapGAN, we first found that using ResBlock as the style conversion module fails to establish long-distance relationships between features. In addition, the small receptive field of convolution layers in ResBlock leads to poor global information capture, resulting in inferior image restoration during upsampling. Second, using transpose convolution as the upsampling method can result in the issue of blurred content in the generated maps. To address these problems, we propose corresponding improvements: on one hand, a style conversion module combining multi-headed self-attention (MHSA) with residual modules, named MHSA-ResBlock, is introduced to address the difficulty in capturing long-distance relationships between features when dealing with a large number of pixel features, and to better capture global information in images. On the other hand, an upsampling method combining transpose convolution with the CARAFE upsampling operator, named TC-Carafe, is proposed to tackle the issues of content loss and blurring associated with traditional transpose convolution upsampling. Furthermore, experimental results show that the MHSA-ResBlock establishes inter-pixel feature relationships and leverages the advantages of fine-grained upsampling operations with TC-Carafe, thereby utilizing inter-pixel feature relationships and neighborhood information to further improve the quality of map generation. Compared to SmapGAN, our research method has shown improvements of 0.6133 and 0.0042 in PSNR and SSIM, respectively. In addition, it has reduced RMSE by 0.72, outperforming SmapGAN in all metrics.

© 2024 Society of Photo-Optical Instrumentation Engineers (SPIE)
Zhipeng Ding, Ben Wang, Shuifa Sun, Yongheng Tang, Ren Zhuang, and Wenbo Liu "Improved SmapGAN remote sensing image map generation based on multi-head self-attention and carafe," Journal of Applied Remote Sensing 18(1), 014526 (20 March 2024). https://doi.org/10.1117/1.JRS.18.014526
Received: 1 December 2023; Accepted: 29 February 2024; Published: 20 March 2024
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Remote sensing

Data modeling

Education and training

Convolution

Roads

Image segmentation

Semantics

RELATED CONTENT


Back to Top