Tsinghua University took over and YOLOv10 came out: the performance was greatly improved and it was on the GitHub hot list

The benchmark YOLO series of target detection systems has once again received a major upgrade.

Since the release of YOLOv9 in February this year, the baton of the YOLO (You Only Look Once) series has been passed to the hands of researchers at Tsinghua University.

Last weekend, the news of the launch of YOLOv10 attracted the attention of the AI ​​industry. It is considered a breakthrough framework in the field of computer vision and is known for its real-time end-to-end object detection capabilities, continuing the legacy of the YOLO series by providing a powerful solution that combines efficiency and accuracy.

Paper address: https://arxiv.org/pdf/2405.14458

Project address: https://github.com/THU-MIG/yolov10

After the new version was released, many people have conducted deployment tests and the results are good:

YOLO has always been the leader in the field of real-time target detection because of its powerful performance and low consumption of computing power. Main paradigm. The framework is widely used in various practical applications, including autonomous driving, surveillance and logistics. Its efficient and accurate object detection capabilities make it ideal for tasks such as identifying pedestrians and vehicles in real time; in logistics, it helps with inventory management and package tracking, and its AI capabilities help people improve efficiency in many tasks.

Over the past few years, researchers have explored YOLO’s architectural design, optimization goals, data enhancement strategies, etc., and have made significant progress. However, post-processing's reliance on non-maximum suppression (NMS) hinders end-to-end deployment of YOLO and adversely affects inference latency. Furthermore, the design of individual components in YOLO lacks a comprehensive and thorough examination, resulting in significant computational redundancy and limiting the capabilities of the model.

The breakthrough of YOLOv10 is to further improve the performance-efficiency boundary of YOLO in terms of post-processing and model architecture.

To this end, the research team proposed for the first time consistent dual assignment (consistent dual assignment) of YOLO without NMS training, which makes YOLO better in terms of performance and inference latency. improved.

The research team proposed an overall efficiency-accuracy-driven model design strategy for YOLO, fully optimizing each component of YOLO from the perspectives of efficiency and accuracy, greatly reducing computing overhead and Enhanced model capabilities.

Extensive experiments show that YOLOv10 achieves SOTA performance and efficiency at various model sizes. For example, YOLOv10-S is 1.8x faster than RT-DETR-R18 at similar APs on COCO, while significantly reducing the number of parameters and FLOPs. Compared with YOLOv9-C, YOLOv10-B has 46% less latency and 25% fewer parameters with the same performance.

Method introduction

In order to achieve overall efficiency-accuracy-driven model design, the research team started from Improvement methods are proposed in terms of efficiency and accuracy.

In order to improve efficiency, this study proposes a lightweight classification head, spatial-channel (spatial-channel) decoupled downsampling and sorting guidance block design to reduce obvious computational redundancy. and achieve a more efficient architecture.

In order to improve the accuracy, the research team explored large kernel convolution and proposed an effective partial self-attention (PSA) module to enhance model capabilities at low Unlock the potential for performance improvements at low cost. Based on these methods, the team successfully implemented a series of real-time end-to-end detectors of different scales, namely YOLOv10-N/S/M/B/L/X.

Consistent dual allocation for NMS-free training

During training, YOLO typically utilizes TAL for each Instance allocate multiple positive samples. The one-to-many allocation approach generates rich supervision signals that facilitate optimization and enable the model to achieve superior performance.

However, this requires YOLO to rely on NMS post-processing, which results in sub-optimal inference efficiency when deployed. While previous research works have explored one-to-one matching to suppress redundant predictions, they often introduce additional inference overhead.

Unlike one-to-many assignment, one-to-one matching assigns only one prediction to each ground truth, avoiding NMS post-processing. However, this leads to weak supervision, so that the accuracy and convergence speed are not ideal. Fortunately, this deficiency can be remedied by one-to-many allocation.

The "dual label allocation" proposed in this study combines the advantages of the above two strategies. As shown in the figure below, this research introduces another one-to-one head for YOLO. It retains the same structure and adopts the same optimization goals as the original one-to-many branch, but utilizes one-to-one matching to obtain label assignments. During training, the two heads are jointly optimized to provide rich supervision; during inference, YOLOv10 discards the one-to-many head and utilizes the one-to-one head to make predictions. This enables YOLO to be deployed end-to-end without incurring any additional inference costs.

##Overall efficiency-accuracy-driven model design

Except for the last In addition to processing, YOLO's model architecture also poses a huge challenge to the efficiency-accuracy trade-off. Although previous research efforts have explored various design strategies, a comprehensive examination of the various components in YOLO is still lacking. Therefore, the model architecture exhibits non-negligible computational redundancy and limited capabilities.

The components in YOLO include stems, downsampling layers, stages with basic building blocks, and heads. The author mainly performs efficiency-driven model design for the following three parts.

1. Lightweight classification head
2. Spatial channel decoupling downsampling
3. Ranking-guided module design

In order to achieve accuracy-driven model design, the research team further explored large-kernel convolution and self-attention Force mechanism is designed to improve model performance at minimal cost.

Experiment

As shown in Table 1, YOLOv10 developed by the Tsinghua team achieved SOTA performance and End-to-end latency.

The study also conducted ablation experiments for YOLOv10-S and YOLOv10-M. The experimental results are shown in the following table:

As shown in the table below, dual-label allocation achieves the best AP-latency trade-off, and optimal performance is achieved with a consistent matching metric.

As shown in the table below, each design component includes lightweight classification head, spatial channel solution Coupled downsampling and sequencing-guided module design both help reduce parameter count, FLOPs, and latency. Importantly, these improvements are achieved while maintaining excellent performance.

As shown in Table 10, the use of large-core convolution and PSA modules improves the performance of YOLOv10-S by 0.4% AP and 1.4% AP respectively with a minimum delay increase of 0.03ms and 0.15ms. Significantly improved.