Accurate extraction of urban impervious surface data from high-resolution imagery remains a challenging task because of the spectral heterogeneity of complex urban land-cover types. Since the high-resolution imagery simultaneously provides plentiful spectral and spatial features, the accurate extraction of impervious surfaces depends on effective extraction and integration of spectral–spatial multifeatures. Different features have different importance for determining a certain class; traditional multifeature fusion methods that treat all features equally during classification cannot utilize the joint effect of multifeatures fully. A fusion method of distance metric learning (DML) and support vector machines is proposed to find the impervious and pervious subclasses from Chinese ZiYuan-3 (ZY-3) imagery. In the procedure of finding appropriate spectral and spatial feature combinations with DML, optimized distance metric was obtained adaptively by learning from the similarity side-information generated from labeled samples. Compared with the traditional vector stacking method that used each feature equally for multifeatures fusion, the approach achieves an overall accuracy of 91.6% (4.1% higher than the prior one) for a suburban dataset, and an accuracy of 92.7% (3.4% higher) for a downtown dataset, indicating the effectiveness of the method for accurately extracting urban impervious surface data from ZY-3 imagery.