A color remote image building extraction method is proposed based on vegetation likelihood modeling and shadow. Gaussian mixture model（GMM） is used to model offline vegetation color features to eliminate the effect of vegetation in RGB color space; mean-shift algorithm is used to extract building shadow areas on the basis of over-segmenting the image

and appropriate post-processing is used to optimize the extraction results; candidate regions are determined by calculating the shadow area pixel. After that

quadratic discriminant analysis（QDA）classifier is used to classify images to implement extracting building ultimately. Experiment results show that the proposed method can improve the accuracy of building extraction effectively.