Synthetic aperture radar (SAR) aircraft detection methods based on deep learning have become a current research hotspot. However, considerable challenges still remain due to the scattering feature of aircraft, variations in aircraft size, and interference from complex scenarios. To tackle these problems, the scattering feature extraction and fuse network (SFEF-Net) is proposed. First, considering the scattering characteristics of aircraft, we propose a scattering feature extraction and relation enhancement (SFERE) backbone based on the deformable convolution and the global context block. The SFERE backbone is used to extract the scattering feature of aircraft and model the correlation of scattering points. Furthermore, to enhance the detection ability for multi-scale aircraft targets in complex scenes, we redesign an attention bidirectional feature fusion pyramid (ABFFP). Two novel modules are proposed in ABFFP, namely, the attention guidance feature fusion (AGFF) module and the residual efficient channel attention (RECA) module. The AGFF module is proposed to suppress the interference of backgrounds and aggregate the multi-level feature maps. After the feature fusion operation, the output feature maps contain richer channel information, but there is some redundant information that could reduce the accuracy. Therefore, we adopt the RECA module to further select useful information in the channel dimension. To demonstrate the effectiveness of SFEF-Net, SAR aircraft images from the Gaofen-3 system are utilized in the experiments. The detection results show that the proposed model achieves competitive performance with an average precision of 95.5%.