Applications of the intensity-hue-saturation (IHS) based image fusion techniques in resource inventory and environmental monitoring are usually hampered by considerable spectral distortion in the spatially enhanced image. The image transform model needs to be regulated via a proper design of the weight structure and controlling parameters to achieve a better spectral fidelity. Use of localized weight estimation and an output constraint to modify the generalized intensity-hue-saturation transform (GIHS) for the purpose of rectifying digital numbers of the fused image back to their original counterparts are proposed. The weight localization was achieved via land cover classification of the multispectral data, and the spectral constraint was constructed using a ratio between individual spectral bands stratified with each land cover type and the modified image intensity value. This method was compared both spatially and spectrally with the traditional IHS and the GIHS that has a weight structure induced from the sensor’s spectral response characteristics. Experiments with WorldView-2 multispectral and panchromatic data indicated that the new image fusion approach achieved the highest level of spectral fidelity with enhancement of spatial details comparable to the other IHS-based methods.