2.1

Motion objects detection

Aiming at the relationship between the upward movement characteristics of smoke and fixed surveillance cameras, the moving target detection method under static background is used to extract the moving area. The Vibe algorithm introduces the randomization method into the background model for the first time, which is fast, has a small calculation and a certain robustness to noise. The detection area is more complete, and it can quickly achieve background extraction and moving target detection [15].

However, the original Vibe algorithm has the following problems when extracting the motion area: the algorithm is updated too fast, and sometimes the initial smoke movement is slow, which will absorb the smoke area as the background, resulting in no smoke detected; if the algorithm is updated too slowly, it will leave shadow; misclassification of pixels may be caused due to parameter setting issues. Aiming at the problems existing in the original Vibe algorithm, the improved algorithm is proposed.

Firstly, the sampling range is extended from 8 neighborhoods to 12 neighborhoods. Enlarging the sampling range can avoid a pixel from being sampled multiple times and improve the quality of the initial model.

Figure 1.

12 neighborhoods model

The original Vibe algorithm is performed by calculating the current pixel and the sample set the Euclidean distance D between each pixel for foreground and background segmentation. As is shown in Figure2, set v(x) (current point)as the center, R as radius, v₁, v₂, v₃…v_n is grayscale values in sample set. Count each distance of sample set with v(x), if the count of distance which is less than threshold T, then take v(x)as foreground, otherwise, take it as background. In original Vibe algorithm, R = 20, T = 2, n = 20.

Figure 2.

Pixel classification in Euclidean distance

On the basis of the original Vibe algorithm, the OTSU is introduced to calculate the optimal segmentation threshold of the current frame, and the foreground points are double-discriminated [16]. OTSU is defined as:

Where the subscript 0, 1 and without subscript indicate foreground, background and entire image, respectively. ω indicate the proportion of the number of pixels, μ are the average grayscale value. Set the segmentation threshold be t, and obtained when the σ² reaches the maximum.

In the detection results of the traditional vibe algorithm, N background pixels are randomly selected, the average gray value V_N of these background points is calculated, and the secondary discrimination is performed:

Secondly, a dynamic threshold is proposed: taking the average distance meanD(x, y) from each pixel to the background sample set first. The dynamic threshold is calculated as follows:

In this paper, α=0.5, α₂ =0.25, β =3, r is original radius parameter 20. The larger the meanD(x, y) is, the more complex the background is, so a larger R is needed, otherwise a smaller R is required. The parameter β is adjusted to an appropriate value according to the background change, and take β meanD(x, y) as the threshold judgment standard.

After morphological processing operations such as dilation, erosion, opening operation and closing operation, the detection result of smoke video motion area is shown in the figure3.

Figure 3.

The result of motion detection

The motion area extracted by the motion detection method includes not only the smoke diffusion area, but also may moving vehicles, walking person, etc. Therefore, further segment is needed to obtain a more accurate smoke motion area.

2.2

Color Extraction

The color attributes have an important role in smoke detection. Color is a powerful descriptive factor in human vision, it carries a large amount of visual information. Although the smoke area has been determined by the motion extraction, there have non-smoke motion areas in these areas that are different in color with smoke. It would cost much time if feature extraction is performed directly on this result. Therefore, color segmentation is performed on the motion region. CIELAB [17] is a color system of CIE that is used to confirm the numerical information of color. It can express colors based on human vision system. The CIELAB values cannot be obtained directly from RGB, but can be obtained roughly from sRGB [18]:

Among this, X_n =0.950456, Y_n =1, Z_n =1.088754.

f(X), f(Y), f(Z) can be expressed as the filter:

By extracting and analyzing the color components of the smoke image in CIELAB color spaces, a color statistical model based on the Lab components is proposed.

C stands for chroma which indicates the vividness of the color.

After using the CIELAB color filtering to segment the suspected smoke area, although most of the interference between the color and the smoke moving area can be eliminated (see Fig 4 and 5), there may still be very few moving areas similar to the smoke color. Therefore, in-depth analysis based on more essential characteristics of the smoke image should be carried out.

Figure 4.

Image before and after color filtering

Figure 5.

Suspected smoke area before and after filtering

2.3

Texture detection

Texture feature is a global feature, which reflects the organization of slow transformation and periodic change on the surface of the object. Better results can be achieved by texture feature in recognizing smoke image. LBP and GLCM are the most widely used method in texture feature extraction.

The earliest LBP descriptor was proposed by Ojala et al. [19] for the texture image classification. Taking the gray value of the center pixel as a threshold for binary quantization, then the neighborhood gray value is encoded as 1 if it is greater than or equal to the gray value of the center point, otherwise, it will be encoded as 0. Then, the LBP value of center pixel is obtained by transforming the 8-bit binary number into decimal number. In the meanwhile, there will be 2⁸=256 patterns.

The uniform LBP [20] was proposed to achieve effective dimensionality reduction, which is defined as when the cyclic binary number corresponding to a certain LBP has at most two jumps from 0 to 1 or from 1 to 0. The binary corresponding to that LBP is called the uniform class, which is proven to be a powerful texture. The rest of the patterns are grouped into another class, called the mixed pattern class. Therefore, the generated binary codes are classified into 58 uniform patterns and one mixed pattern, for reducing computational complexity.

GLCM [21] can reflect the comprehensive information of image gray scale, such as direction, adjacent interval and amplitude of change, but it cannot directly provide the features of different textures. Therefore, statistical parameters that can quantitatively describe texture features are extracted by counting the values of normalized GLCM.

Where (i, j) is the grayscale value of any point(x, y) in image and another point(x + Δx, y + Δy), their distance is d and the angle is θ, let the level of gray value be g. For the whole image, count the number of occurrences of every (i, j) and arrange them into a square matrix. Normalize the occurrences to get the probability, and P(i, j,d,θ) is obtained. Among this, , , , .

In this paper, energy characteristics, entropy, correlation and contrast, 4 directions and 16 features vectors are selected to further identify smoke.

3.1

Experiment configuration and evaluation index

The first experiment for comparing between different texture features and different classifiers was first conducted based on Windows 11 system with Intel Core i5 CPU; the software is Matlab2018. In the experiment, the smoke dataset from the National Laboratory of Fire Science, University of Science and Technology of China is used [22]. It includes 800 still images of 100×100 pixels, dividing into two groups, 400 images containing smoke and 400 images not containing smoke. 59 uniform LBP patterns and 16 GLCM paraments, totally 75 feature vectors were extracted and feed into a classifier for training to obtain a classification model. SVM and random forest (RF) were compared.

In order to measure the performance of the algorithm quantitatively, the confusion matrix is chosen as the index to evaluate the retrieval performance of the algorithm, TN denote the number of samples which is predicted to be negative from smokeless, FP denote the number of samples which is predicted to be positive from smokeless, FN denote the number of samples which is predicted to be negative from smoke and TP denote the number of samples which is predicted to be positive from smoke. Accuracy (Acc), precision (Prec), recall rate (Rec) and the value of in (13)-(16) is used as measures for testing algorithms.

3.2

Experiment results

Three different features, LBP, GLCM and the combination of LBP and GLCM, and two classifiers, SVM and random forest (RF), were compared in the experiment. 400 random images, half with smoke and half without smoke, were selected as the training images and test images respectively. The comparison result is presented in Table I and Figure 6 for 3 texture features and two classifiers. In Table I and Figure 6, LG denotes the combination of LBP and GLCM features, RF denotes random forest.

Figure 6.

Comparison of evaluation index of classification

Table 1.

Evaluation of different method

It can be seen from Table I that the Acc, Prec, Rec, and value of SVM is better than those of RF when only classifying GLCM features. However, when it comes to LBP and the combination of LBP and GLCM, the result of RF is better than SVM. By analyzing the characteristics of the SVM, it can be seen that when there are too many features, it is difficult to find an optimal hyperplane to divide two types of samples, the reliability of segmentation is not enough, and the generalization ability is insufficient. Therefore, SVM has better effect on small sample training, but if the number of image features is large, the classification effect is poor. By contrast, RF has a high degree of self-adaptation and good generalization ability.

In the second experiment, the RF classifier that has been trained is used to test smoke video. In the experiment, the proposed Vibe algorithm and color filter were implemented to segment suspected smoke area, and then combination of LBP and GLCM features were extracted and feed into trained RF classifier. The results were compared that from with Liu’s method [23] as is shown in the Table II. It used background modeling and color information to mix the static and dynamic features, then SVM was used to discriminate texture features. On the whole, the performances of the proposed method outperform Liu’s method. Liu’s algorithm is easy to be influenced by surroundings, and some object with subtle texture, blurred color and irregular movement are likely to be detected as smoke.

Table 2.

Smoke video detection results