A spectral–spatial classification method using a trilateral filter (TF) and stacked sparse autoencoder (SSA) for improving the classification accuracy of hyperspectral image (HSI) is proposed. The operation is carried out in two main stages: edge-preserved smoothing and high-level feature learning. First, a reference image obtained from dual tree complex wavelet transform is adopted in a TF for smoothing the HSI. As expected, the filter not only can effectively attenuate the mixed noise (e.g., Gaussian noise and impulse noise) where the bilateral filter shows poor performance but also can produce useful spectral–spatial features from HSI by considering geometric closeness and photometric similarity between pixels simultaneously. Second, an artificial fish swarm algorithm (AFSA) is first introduced into a SSA, and the proposed deep learning architecture is used to adaptively exploit more abstract and differentiable high-level feature representations from the smoothed HSI, based on the factor that AFSA provides better trade-off among concurrency, search efficiency, and convergence rate compared with gradient descent and back-propagation algorithms in a traditional SSA. Finally, a random forest classifier is utilized to perform supervised fine-tuning and classification. Experimental results on two real HSI data sets demonstrate that the proposed method generates competitive performance compared with those of conventional methods.