In measuring a transmission matrix (TM) of a scattering medium, the normal way requires projecting a large number of patterns, resulting in significant measurement time. Here, we propose to do imaging through scattering media using a path-planning-based downsampling TM. The proposed method is verified by measuring TMs for imaging through scattering media when Walsh, Cake-cutting, and Russian doll Hadamard transforms (HTs) are adopted. Based on the spectral characteristics of different HTs of natural images, corresponding sampling paths are artificially planned, which guarantees the method to preferentially measure components in TM in relation to the more important spectra. This means that our method can reduce the number of patterns during measurement, thus improving the imaging speed while only sacrificing a little imaging quality. Experimental results demonstrate the effectiveness of the proposed method.