A novel shadow detection method for color remotely sensed images that satisfies requirements for both high accuracy and wide adaptability in applications is presented. This method builds on previously reported work investigating the shadow properties in both red/green/blue (RGB) and hue saturation value (HSV) color spaces. The method integrates several shadow features for modeling and uses a region growing (RG) algorithm and a perception machine (PM) of a neural network (NN) to identify shadows. To ensure efficiency of the parameters, first the proposed method uses a small number of shadow samples manually obtained from an input image to automatically estimate the necessary parameters. Then, the method uses the estimated threshold to binarize the hue map of the input image for obtaining possible shadow seeds and applies the RG algorithm to produce a candidate shadow map from the intensity channel. Subsequently, all of the hue, saturation, and intensity maps from the candidate shadow map are filtered with a corresponding band-pass filter, and the filtered results are input into the PM algorithm for the final shadow segmentation. Experiments indicate that the proposed algorithm has better performance in multiple cases, providing a new and practical shadow detection method.