1.

Introduction

With the development of highway and rural roads, road networks have recently become more and more complex, and many new roads are constructed every day. In order to utilize and manage the roads’ network effectively, sketching the roads’ network based on aerial images is a significant task, but it is difficult to make sure that all aerial images are clear and of good quality. In addition to the influence of the changing weather on image acquisition, automatic road detection and identification can be greatly affected by complex infrastructures, farmland, and mountains.

Fischler did road detection with multisource aerial images 30 years ago. His detection could only be done on low-precision images but was not available for high resolution images. Later, many researchers provided the different detection methods in this topic.¹ Wen and Wang² studied an algorithm to automatically extract the main objects on roads from remote sensing images in 2000 where the roads are wide. One year later (2001), Cai and Wang³ proposed an algorithm for extracting main road networks from multiband remote sensing images. In 2004, Zhu et al.⁴ processed high resolution remote sensing images to get the roads and their details based on morphological segments,⁴ but not for thin roads. In 2007, Yang and Wang⁵ used the scene perception of remote sensing images and image classification to detect and analyze roads. In the same year (2007), Luo et al.⁶ suggested the method of combining spectrum and shape features to extract roads for clear images. Also, in 2007, Hu et al.⁷ did some related work based on aerial image characteristics; they analyzed road networks mainly by using the crossings and corners of the detected roads. Cem and Beril⁸ studied road network extraction from high resolution aerial images by using probabilistic and graph theoretical methods. In the past 30 years, this topic has become hotter, but there is no standard algorithm for detecting thin roads or identifying roads in vague aerial images under bad weather.

When the weather is bad, aerial images are blurred. One hard main task is to enhance aerial images of poor quality, because bad weather often causes aerial or remote sensing images to be blurred and reduces visibility. On the basis of the existing systems, the image enhancement is the foundation for solving this kind of problem. There are many methods to process complex and vague images, and one of them is presented by Eitan. He studied an algorithm by using hierarchy and an adaptive idea in segmenting visual scenes. This method can accurately segment butterflies from the background of similar textures and colors in grass, or tigers and leopards from the background of similar rock patterns, and so on.⁹ In 2007, Lawrence¹⁰ described a new image enhancement method which emphasized the importance of image enhancement. It stated that it was essential to enhance and process images according to image features. In 2007, Oakley and Hong¹¹ published a paper about a method of contrast correction in color images, especially for fog images. Pu et al. also studied the detection of weak texture information by using a fractional differential.^12–14 However, all of the above image enhancement algorithms were not for thin and vague road detection in aerial images.

Due to the low contrast of aerial images taken in bad weather or in complex situations, no matter whether they are among objects or between objects and the background, the thin or vague roads with a low contrast in gray scale (luminance) or a small color difference are easily ignored by a normal image processing method. Also, a slender road image is often mixed in with a variety of natural features and man-made facilities; sometimes, it is even difficult to identify these with human vision. In this case, a complex road network is hard to identify through the approach that is based on gray scale similarity.¹⁵ It is also difficult to detect ridge-like roads by using a conventional edge detection method and even when applying an improved smart Canny edge detection algorithm,¹⁶ some thin and small lines or curves cannot be completely detected in most cases. In general, roads in aerial images or remote sensing images are ridge-like linear objects, so developing a special ridge edge detection algorithm for the images is important. Wang¹⁷ studied a valley edge detecting algorithm for froth images. It is valuable for froth images because the valley edges in an image can be roughly detected by one threshold. The road edges in an aerial image cannot be detected by using only one threshold because of the effects from different objects. Hence, when Wang’s algorithm is used for thin or small road detection in an aerial image, it is hard to determine a correct gradient threshold: a low threshold will produce a lot of noise and a high threshold will lose useful information which influences road identification. Therefore, the studied method in this paper is first to enhance ridge edges (roads) based on a fractional differential, then detect thin or vague roads with a newly designed ridge edge detecting algorithm. Of course, for the whole method of detecting thin or vague roads, although ridge detecting is a significant step, more post functions are needed for the final road identification.

2.

Ridge Edge Detection Algorithm for Thin and Vague Roads

In aerial images, there are complex crisscross road lines; sometimes they are clear, sometimes they are not clear due to sunlight, aircraft jitter, or complex ground scenes. In the unclear cases, the image processing technology is required to enhance images for detecting roads as accurately as possible. For a low precision image, a road maybe a ridge or valley line (curve) with a certain width, whereas for a high precision image, in addition to the above properties, the solid and dashed lines in a road are also ridge edges. In most aerial images, there are woods, roads and railways, grass and crops, residential buildings, rivers, and mountains, etc., which lead to the complexity of ground scenes. Hence the roads are difficult to detect by a conventional image processing method. In most cases, for such a complex image, the first- and the second-order differential operators cannot accurately detect all roads, and the operators often extract only the high contrast objects and partial roads as shown in Fig. 1 (gray scale image) and Fig. 2 (color image), where, though the roads are ridge edges, some nonroad objects are similar to the roads. These features are clearly observed on the gray level histogram of the image section profile, and this is why traditional edge tracing algorithms can only detect partial roads that have obvious edges. So, a new ridge edge detection algorithm is studied in this study according to the above characteristics.

Fig. 1

Section profile and its gray level histogram in an aerial gray scale image.

Fig. 2

Section profile and its histogram in an aerial color image.

In Fig. 1, the diagonal black line in the left picture is a section profile line, and from bottom to top its gray level histogram is shown in the right picture. In the left picture, the two vertical green lines correspond to the end points of the profile line, and in the histogram, from left to right, the five gray level peaks represent the five roads (see the arrows in the histogram corresponding to the arrows from bottom to up in the left picture).

When analyzing the above information, without connecting with the other objects with the same gray scales, the roads are ridge edges (such as the first and the fifth roads), which are easily detected. But a road is difficult to detect when the gray scale value of the ridge edges is too low, e.g., the fifth road. If a road is connected to or close to neighboring similar objects with the similar gray scales, the peaks of the histogram show the rectangles or closely linked multiple triangles, which creates difficulty for road detection.

The profile line in Fig. 2 (left picture) intersects (from bottom to top) with eight roads with ridge edges (see the arrows from left to right in the histogram corresponding to the arrows from bottom to up in the left picture). In fact, 24 obvious peaks are shown in the histogram, but it is difficult to judge the corresponding roads due to noise.

In order to detect these ridge edges (Figs. 1 and 2), this paper particularly designs a new image edge detection algorithm. It includes several image processing subalgorithms. The result of edge detection is a binary image without using any thresholding operation. The detecting rules are: (1) For each detecting point, in the four directions that are on horizontal, vertical, and the two diagonals, the algorithm checks if the point is a ridge edge point. If this is true, it is chosen as one ridge edge point candidate; (2) To avoid the impact of noise, the gray scale value of the candidate point must be higher than that of the weighted average value of neighborhoods in the same direction; (3) To finalize the detection result, a number of postprocessing functions are studied, such as deleting isolated points, removing extra lines and points through a thinning operation and a length thresholding function, and detecting the remaining candidate points with a $7\times 7$ template [since the short line is of less than 3-4(7/2) pixels] to remove the noise points or short lines/curves. The ridge edge detection procedure is presented as follows.

For each of the detecting points, the algorithm measures its magnitude gradient values in the four directions, respectively, and determines which direction is most likely to be a ridge edge point orientation. The detection kernel size can be $5\times 5$ when a road width is less than 6 pixels; the weight of each point in the kernel is inversely proportional to the distance from the detecting point. An example of a $5\times 5$ kernel is illustrated as follows (see Fig. 3). Assuming an image is $f(i,j)$, and the current detecting point is $(i,j)$. Figure 3 is for the detecting kernels in three different directions respectively: Figs. 3(a), 3(b), and 3(c), respectively, represent 0 deg (90 deg is similar), 45 deg, and 135 deg detecting kernels. There are five straight lines in each of the kernels, $L$ is in the middle line, above it we have $L_1$ and $L_2$, and under it we have $L_{-1}$ and $L_{-2}$. Each grid in the figure represents a pixel and the label in each grid is for the pixel location in the image.

Fig. 3

Three detection kernels in three different directions: (a) 0-deg direction; (b) 45-deg direction; and (c) 135-deg direction.

As the roads are straight lines or curves, using several pixels instead of a single pixel for the detection can increase the accuracy of the detection result. In a $5\times 5$ template, in the detection procedure for each kernel, the algorithm detects 3 to 4 pixels in a line as shown in Fig. 3, rather than detecting an isolated pixel,^17,18 which can reduce noise. The postprocessing is for identifying the candidate lines. As an example of a $5\times 5$ template for an image $f(x,y)$, the detecting rules in four directions can be presented as

In the above equations, the situation in 135 deg is taken as an example. If $F_1>0$; $F_2>0$; $F_3>0$; $\begin{array}{c}{F}_4>0\end{array}$, the algorithm calculates $T_{135}=\sum _1^4{\omega }_i{F}_i$, where, ${\omega }_i$ ($i=1$, 2, 3, 4) is the weight variable, such as set ${\omega }_1={\omega }_2=1.3$, ${\omega }_3={\omega }_4=0.7$. In the same way, $T_0$, $T_{45}$, $T_{90}$ can be obtained, too. $T$ is a preset threshold, and as $T$ increases, the number of ridge points decreases. If $T_{\max }(\begin{array}{c}{T}_0,T_{45},T_{90},T_{135}\end{array})\ge T$, this point is a candidate ridge point and its direction is marked. In the candidate points, many noise points may exist. To remove these noise ridge points, the following postprocedure is suggested.

First, the algorithm connects the near neighboring ridge points in the consistent direction, then removes some noise or isolated ridge points. After finding a short curve or a short line (e.g., less than 4 pixels), its two endpoints are detected. After that, the possible forward direction for each endpoint is estimated, and then in each direction, its two or more neighborhood points are smoothed as a line and its new direction is obtained. To easily understand the above algorithm, the algorithm working steps and the detection result are shown in Fig. 4.

Fig. 4

Ridge edge detection working steps: (a) original binary image; (b) end point (red) and junction point (green) detection; (c) small gap linking and noise removal; and (d) large gap liking and curve smoothing.

Figure 5(a) presents a fuzzy aerial image including several roads. By comparing the above new algorithm to the four conventional edge detection algorithms, the testing results are shown in Figs. 5(b)–5(d). The Canny algorithm gives the binary image result which is similar to that obtained by using the Sobel algorithm. It also includes many noisy lines and partial bifilar lines, and creates troubles for detecting road locations accurately. The LOG algorithm can detect the main roads, but will miss some small or thin roads. The new ridge detection algorithm can detect all the roads in the image very well with a binary image result, unlike the LOG result which only presents the gradient images and also needs an appropriate dynamic threshold to turn a gradient image into a binary image. Although the result of the Canny algorithm is a binary image, it needs to predetermine two thresholds to make gradient image binarization. In addition, the road detected by the new algorithm is of one pixel width (not a bifilar line), so it can give the exact location of a road. Although the detection result of the studied algorithm includes some noise, most of noise can be filtered away according to predetermined thresholds of the length or area. The image in Fig. 6 is for a city block image, which is more complicated than that in Fig. 5; the Canny edge detector cannot detect all the roads, but when the new algorithm is used on it, the roads can be very well detected. After primary edge detection, nonroad edges or textures need to be filtered away. The noisy edges can be smoothed away by the procedure in Sec. 3.

Fig. 5

Comparison between three edge detectors for a vague aerial road image: (a) original aerial image; (b) Canny edge detection; (c) LOG edge detection; and (d) ridge edge detection.

Fig. 6

Edge detection for complicated and multiple roads in an aerial road image: (a) original image; and (b) ridge edge detection on (a).

3.

Thin and Vague Road Identification in Aerial Images

After wide roads are detected by using other methods, thin or vague roads can be detected by the above new algorithm. To apply the above ridge edge detection algorithm to aerial images for thin and vague road detection and identification, it also needs to merge other image processing functions and algorithms in image preprocessing and postprocessing. In this way, the whole road detection method is constructed. Since the detected roads are thin lines or curves, the traditional algorithms are difficult to use. Figure 7 shows the workflow of the proposed method procedure.

Fig. 7

Work procedure flowchart of ridge edge detection method.

Fig. 8

Tiansi kernel: (a) kernel represented by fractional differential order; and (b) digital kernel.

In experiments, a number of images are tested. In the following, the three typical representative aerial images and their testing results are presented. In the three images, one (Fig. 9) is a simple aerial image where several roads intersect and the image background is not complicated; the second image in Fig. 10 is a farmland image where the image surface is rough with different colors and textures; and the third image in Fig. 11 is more difficult to deal with than the others since it is more complicated. The proposed algorithm is compared to the conventional edge detection operators as above mentioned, but here the new algorithm is mainly compared to the Canny algorithm, because the final results of the two algorithms are binary images, not gradient magnitude images, and the Canny algorithm is the best one of the most conventional algorithms.

Fig. 9

Single road extraction by using new algorithm on image #1: (a) aerial road image #1; (b) ridge edge detection; (c) noise removal and gap link; and (d) final identification result.

Fig. 10

Image enhancement and road identification for complicated aerial road image #2: (a) original image #2; (b) result by fractional differential; (c) Canny edge detection result on (b); (d) new algorithm result on (b); (e) road identification at middle stage; (f) road identification at final stage.

Fig. 11

Complicated and multiple road detection and identification in an aerial image #3: (a) original image #3; (b) result by fractional differential; (c) Canny edge detection result on (a); (d) ridge edge detection on (b); (e) road identifying at middle stage; and (f) road identifying at final stage.

Figure 9(a) is for an aerial image with obvious roads, and the roads have similar colors as that of farmlands and their surrounding block objects, in which case the roads are hard to detect only based on colors or gray scales. The image histogram is shown as in Fig. 1.

In this situation, it seems that the result of the fractional differential algorithm shows the clear roads [Fig. 9(b)] without too much noise, unlike the traditional algorithms (e.g., the Canny algorithm). A more satisfactory result can be obtained with the new ridge edge detection algorithm and the postprocessing functions. Even though the ridge edges [Fig. 9(c)] are mixed with some noise, the noise can be removed according to the length threshold of the lines [Fig. 9(d)], and then the roads are easily extracted by a Hough line detection algorithm.

Figure 10(a) shows a fuzzy aerial image with complex scenes, including roads, houses, bushes, farmlands, forests, rivers, and sea, and its profile histogram features are shown in Fig. 2. To enhance the roads, the image is operated on with the fractional differential operator, and its result is presented in Fig. 10(b): all the roads become clearer. In the Canny edge detection result, partial roads are detected, but more than half of the roads are bifilar lines, which creates difficulty for postprocessing [Fig. 10(c)]. When using the new ridge edge detection algorithm and the road identification procedure (postprocessing) described in Fig. 7, the result is obtained as illustrated in Fig. 10(d), where, by comparison to the Canny edge detection algorithm, less noise is obtained, it shows the main road lines or curves clearly, and the resultant binary image is relatively simple. Therefore, it provides better conditions for postprocessing.

The image in Fig. 11 is a complex urban fuzzy aerial image, including large quantities of artificial facilities; the roads to be detected are not clear. So, first the roads are enhanced by using the fractional differential operator, and then the ridge edges are detected to initially extract partial roads with the new algorithm. In the image, some road information is missing [Fig. 11(c)], probably because of the weakness of the roads. Even for these kinds of images, the new ridge edge detection algorithm can detect most of the road information [Fig. 11(d)]. In Fig. 11(e), the small objects or noise are filtered away from the result of the new edge detection, and the result in Fig. 11(f) is the final road identification result where most roads are distinctly shown after road identifying is complete.

The image in Fig. 12(a) is an aerial image from Ref. 8. The network detected and identified by the algorithm in Ref. 8 is presented in Fig. 12(b), and the result using the new algorithm is illustrated in Fig. 12(c). By comparing the new method to the algorithm in Ref. 8, although the new algorithm only gives a single line (curve) for each road (not for road boundaries), it can produce better road connections in a general structure.

Fig. 12

Algorithm comparison for road identification on a complicated aerial road image: (a) original image from Ref. 21; (b) result of algorithm in Ref. 21; and (c) result of new algorithm.

The difficulty for this new algorithm is shown in Fig. 13. Figure 13(a) is a very blurred and complicated aerial road network image of the same resolution of $2000\times 2000\text{pixels}$ as that in the above images, where there are many buildings and streets among roads. A low threshold $T=1$ for the new algorithm can result in many noise edges, which make road extraction hard. A high threshold $T=6$ can produce less noise, but some road information is lost, which also makes road detection and connection hard.

Fig. 13

Processing for vague and complicated aerial road image: (a) original aerial image; (b) result of edge detection by new algorithm with a low threshold 1; and (c) result of edge detection by new algorithm with a high threshold 6.

4.

Conclusions

This paper suggests a method for resolving the problem of the difficulty in extracting thin and vague road lines or curves from aerial images. Compared to some traditional algorithms, the main features of the new operator are: (1) Enhance images through a fractional differential template to highlight road details,; compared with the conventional Laplacian and high-pass filter operators, the new operator can enhance road information without adding too much noise; (2) Then get road lines or curves by using the new ridge edge detection algorithm based on ridge characteristics proposed in this study. The algorithm is different from general edge detection algorithms, in that it is designed not based on point detection, but based on lines of 3 to 4 points [see Eqs. (1)–(4)], and the detected edges are ridge edges. When comparing this method to the conventional edge detection algorithms (such as Canny, LOG, etc.), the results show that the proposed method can be applied for roughly detecting thin and vague straight and curved roads in high-resolution aerial images with satisfactory results. After edge detection, the road identifying procedure (postprocessing) includes a number of functions, such as Hough line detection, thinning, endpoint detection, endpoints connecting, removing small object noises, nonlinear object removal, etc., which makes the final road identification result good. If the width of roads is wide (say, more than 6 pixels on average), the original image is shrunk, and the detection result will be used to re-map the roads based on both detection results and the original image information.

Currently, a three-dimensional (3-D) laser technique has been used to recognize roads by combining image techniques.^19,20 Therefore, further research will make full use of 3-D texture and color information to improve the fractional differential operator and the ridge edge detection algorithms for images. The postprocedures are very important for road mapping or identifying. A number of functions or algorithms require improvement for different types of aerial road images.