Light detection and ranging (LiDAR) point cloud data can contain millions of point returns from a diverse range of surface features, and directly reconstructing buildings from these data is challenging. Trees and other vegetation pose a particular problem in many built environments. This paper investigates several efficient procedures for detecting buildings and excluding vegetation using LiDAR and imagery data. Two general approaches for identifying and filtering out returns from vegetation are investigated: the first uses a normalized difference vegetation index (NDVI) image, while the second uses height differences. The utility of an entropy filter for improving NDVI filter performance as well as two distinct approaches for height-difference modeling are also evaluated. All methods use efficient raster-based algorithms for filtering while retaining the high spatial precision of the vector LiDAR point returns. Following removal of nonbuilding points, remaining points are segmented into distinct building features. In addition, we place particular emphasis on the analysis of processing challenges and special cases as well as the accuracy of these different methods on a large-volume LiDAR dataset covering a challenging build environment.