Rapid damage assessment after an earthquake is vital for an efficient emergency response. With the rapid development of unmanned aerial vehicles (UAVs), they can now be used to rapidly assess the building damage and have the advantages of real-time operation, flexibility, low cost, etc. However, UAV images are “big data,” and UAVs can obtain hundreds of scene images in a short period of time. It is, therefore, important to speed up the processing time for UAV images. This paper proposes a parallel processing approach for accelerating the speed of automatic three-dimensional (3-D) building damage detection, using a preseismic digital topographical map and postseismic UAV images. From the experimental results obtained from 3-D building damage detection of the 2013 Ya’an earthquake in Baoxing County, Sichuan province of China, the comparison of the parallel processing in terms of digital surface model generation using postseismic UAV images shows that the total cost of the multicores central processing units (CPUs) and graphics processing unit-based implementation is about 11.0 times faster than the single-core CPU implementation.