3.1.1

YOLOv5 Algorithm Principle.

YoloV5s is mainly composed of four parts, as shown in Figure 1, including: Input, Backbone network, Neck, and Precaution. Input mainly preprocesses the data through Mosaic data enhancement, adaptive image filling, and automatic setting of the initial anchor box size. The backbone network is mainly composed of BottleneckSCP¹⁹ and SPP²⁰ (spatial pyramid pooling). BottleneckSCP is used to improve the inference speed and reduce the amount of calculation. SPP is used to extract multi-scale features from the feature map, so as to complete the extraction of different levels of features from the image. The Neck network layer includes path aggregation structure PAN and feature pyramid FPN. FPN uses a top-down approach to convey semantic information in the network, while PAN uses a bottom-up approach to convey localization information, and integrates different information from various network layers in the backbone network to improve the ability of object detection. As the final detection part, target prediction is primarily to predict objects of diversiform sizes on feature maps of different sizes.

Figure 1.

YoloV5 algorithm structure diagram

YOLOv5 fused attention mechanism. For the image based on the roadside information source, the vehicle target is small and occupies less pixels. The Yolov5 model algorithm is insensitive to the vehicle feature information of the small vehicle target during convolution sampling, and the detection effect is poor²¹. The coordinate attention mechanism is introduced in this paper to reconstruct the attention of the final feature map so as to highlight the important and sensitive information in the feature map while suppressing the general information. For small targets and dense targets, the feature information can be effectively extracted and the accuracy class of detection can be further improved. Coordinate Attention (CA)²² adds location information to channel attention, so that the network can pay attention to a larger area. In order to alleviate the problem of location information loss caused by two-dimensional global pooling proposed by previous attention mechanisms such as SENet²³ and CBAM²⁴, we decompose the channel attention mechanism divided into two parallel unidimensional feature encoding systems, which aggregate features in two directions respectively. In one direction, remote dependencies can be obtained, and in the other direction, precise location information can be obtained. The resulting feature maps are encoded to form a pair of direction-aware and position-sensitive features.

As shown in Figure 2, CBAM pays attention to both spatial and channel information, reconstructs the feature map among the total network through two sub-modules CAM (channel attention module) and SAM (spatial attention module), emphasizes important features, suppresses general vehicle features, and achieves the goal of improving the vehicle target detection effect. This paper integrates CBAM behind Backbone and before Neck network. The main reason for this is that YOLOv5s completes the feature extraction process in Back-bone, and then predicts the output on a large number of feature maps after Neck feature fusion. CBAM performs attention reconstruction here, which can play a crucial role in connecting the above and the next. The specific structure is shown in Figure 3.

Figure 2.

CBAM (Caption centered convolution Block Attention Module).

Figure 3.

Algorithm flow chart of Deep SORT.

3.1.2

Loss Function Improvements.

At present, the main application of the minimum target detection based on the Anchor prediction mechanism is to continuously improve the measurement accuracy of the objects in the target prediction frame by measuring the height mean square error distance between the coordinates of the object in the minimization target prediction and the object coordinates of the enlarged target prediction frame. In the original YOLOv5 algorithm, the IoU Loss loss function uses the GIoU Loss loss function. GIoU Loss can also be a distance metric of a loss function, which can directly meet the metric requirements of the basic loss distance function. At the same time, because GIoU Loss also has a strong scale invariance, the expression is as follows:

However, when GIoU Loss contains two boxes, GIOU Loss will degenerate into IOU Loss and GIOU Loss need to iterate many times to converge, considering the shortcomings of GIou, the article introduces CIoU Loss, the expression is as follows:

Among them: α is the weight, 𝑣 measures the similarity of the depth width ratio, b, b^gt separately represent the center point of the prediction frame and the target frame, and the distance between the two adopts the Euclidean distance ρ. c represents the slant distance of the smallest bounding box that can contain both the prediction box and the target box.

CIoU Loss can directly minimize the distance among the central point of the predicted point and the real point to accelerate concentration. At the same time, it also increases the distance loss that can detect different scales of the real frame, and increases the loss of length and width, so that the entire predicted frame will be more completely consistent. real box. So the CIoU Loss in the article replaces the original GIoU Loss, the effect will be better.

3.2.1

Data Association and Matching.

Deep SORT merge appearance information with motion information to match predicted and tracked boxes using a Hungarian algorithm. For motion information, Deep SORT algorithm usually uses Mahalanobis distance to characterize the correlation coefficient between Kalman filter prediction results and detector results, as shown in formula (3):

In the formula, d_j and y_i represent the state vector of the j-th detection result and the i-th prediction result, and S_i represents the covariance matrix among the detection result and the average tracking result. The Mahalanobis distance measures the standard deviation of the detection results from the average tracking results, taking the uncertainty of the state estimation into account, and can exclude low-probability associations.

When the certainty of target vehicle motion information is high, Mahalanobis distance performs well as an appropriate correlation factor. However, when there is target occlusion or camera Angle jitter in the environment, only using Mahalanobis distance correlation will make the identity of the target switch and eventually lead to target loss. Therefore, consider adding appearance information, calculate the corresponding appearance feature descriptor r_j for each detection frame d_j, and set ||r_j|| = 1. For each tracked trajectory , set the feature warehouse L_k= 100, which is used to store the feature descriptors, L, which are successfully associated with the last 100 targets. Calculate the minimum cosine distance always among the i-th tracking frame and the j-th detection frame, as in formula (4):

When d⁽²⁾ (i, j) is less than the specified threshold, the association is considered successful.

Mahalanobis distance can give reliable target location information in the case of short-term prediction. The cosine similarity of appearance features can be used to restore target ID when target occlusion reappears. In order to complement the advantages of the two metrics, a linear weighting method is used. To combine:

3.2.2

Vehicle Re-identification Model.

The vehicle re-identification model calculates the cosine distance between classes through deep learning, trains the deep learning network offline, and obtains the deep learning weight²⁵. The input object is based on the weight of the network, the direction of the decision boundary and the closest distance to find the bunch to which the object belongs, to achieve Tracking object object re-identification (Re-ID), the unification of metric learning and classification, while improving the recognition accuracy²⁶. Using CNN to perform offline training on large-scale reidentification datasets, the CNN network structure is shown in Table 1. The network adopts a wide residual module, all convolution kernels are 3×3 in size, and convolution with stride 2 is used instead of max pooling²⁷. When the spatial resolution is reduced, the number of channels is increased to avoid the bottleneck problem, Use an exponential liner unit (ELU) as the activation function throughout the network. The network structure is shown in Table 1.

Table 1.

Re-identify network structure.

4.2.1

YOLO v5 Vehicle Detection Model Training.

The backbone network of the detector model selects Darknet-53 for feature extraction, and YOLO v5 is trained using the UA-DETRAC dataset. Modify the original label, only keep the Car, van, Truck categories. According to the specific hardware information, the batch size is set to 16, the filter size is adjusted to 36, and the rest of the hyper parameters remain unchanged. The change trend of the loss function in the training process is shown in Figure 4. In the figure, L is the loss value.

Figure 4.

YOLOv5s loss curve.

4.2.2

Vehicle Re-identification Model Training.

The training is based on cosine metric learning, using the constructed vehicle re-identification dataset, setting the batch α size to 32, the learning rate to 0.001, and the balance parameter γ to 0.085. The training results are shown in Figure 5. The training results show that after 400,000 iterations, the classification accuracy Ac is stable at 96.52%. At this time, the vehicle re-identification network has good classification ability and can accurately re-identify vehicles that have disappeared and reappeared in the field of vision.

Figure 5.

Change trend of training accuracy of vehicle rerecognition model.