Stay up late tonight for Messi and Modric! This CV model allows you to win by guessing the ball

The day before yesterday, Ronaldo’s fans were heartbroken.

In the World Cup in Qatar on December 11, Portugal lost to Morocco and stopped in the quarterfinals. Ronaldo's World Cup journey came to a regrettable end.

##C Ronaldo said the dream is over#This hot search has gathered countless heartbroken fans.

The match between Portugal and Morocco had an upset result of 0:1. Ronaldo is crying, Neymar is crying, who will be next?

However, before I can get over the sadness for Ronaldo, tonight we have another duel between Messi and Modric!

Afraid your heart can’t bear it?

Now, with such a model, you may be able to know in advance whether the game will be an upset within the first 5 minutes of the game.

When computer vision is used in football

This foreign machine learning engineer has always hoped to combine computer vision (CV) and Combined with his passion for football.

Three years ago, he tried to use YOLOv3 to detect and classify players on the basketball court.

The current FIFA World Cup made him pick up this idea again.

This time, he used a combination of YOLOv5 and ByteTRACK to track football players on the field.

Step 1: Build a pre-trained detector

Initially, Skalski hoped to skip custom model training.

He hoped that it would only take a few hours from start to finish and be a quick victory.

Because the popular COCO data set used to train YOLOv5 and YOLOv7 models contains the two classes he is most interested in - person and sports ball, this makes him very optimistic and thinks that it can be COCO checkpoint for training.

He quickly downloaded dozens of short football match videos from Kaggle’s Bundesliga data war and used the pre-trained model for testing.

After trying various configurations such as iou_threshold, confidence_threshold, input_resolution, etc., he got disappointing results.

On this model, the ball is only detected on a few video frames, because there are too few for the model to reliably track such a small and fast-moving object.

#On the other hand, this model will detect many redundant objects outside the field - coaches, fans, maintenance personnel, camera crews, etc. For these redundant objects, additional detection and filtering logic needs to be written, which is too troublesome.

It can be seen that even if the dataset contains the required classes, the videos used for inference are definitely very different from the images used for training.

In this case, Skalski can only create its own dataset and train a custom model.

Step 2: Build a custom detector

Training a custom model requires a lot of effort.

The main reason is that a lot of manual work has to be done in order to annotate images.

Due to the rush of time, the process needs to be simplified as much as possible.

Skalski selected a few dozen videos from the aforementioned Kaggle dataset and used FFmpeg to extract 10 frames from each video - one every 3 seconds.

The next step is to perform automatic preliminary annotation on the image and save the detected objects to a txt file to utilize the pre-trained YOLOv5 model.

Next, the second stage is to improve the data.

Although the computer had completed 90% of the work, manual corrections still took Skalski nearly five hours.

At the same time, he also introduced two additional object classes - referee and goalkeeper. The results of his work can be seen on Roboflow Universe. The dataset is open source and free for anyone to use.

Pre-trained vs custom football player detector

Custom The training results make Skalski confident in solving the above two problems.

But unfortunately, the model still has problems in detecting the goalkeeper class, possibly due to classification imbalance.

So, in the next step, Skalski treats every goalkeeper as an ordinary football player.

Skalski’s model is available on Roboflow Universe, as is his dataset. Just drag and drop the test image.

Step 3: Track the Object

Skalski is using ByteTRACK (SOTA Multi-Object Tracker 1) to track objects in videos.

ByteTRACK does not use any neural network internally. The power of this model comes entirely from simple math: comparing bounding box positions across frames.

The most important thing is that ByteTRACK is not responsible for detection, only tracking. Therefore, it does not require separate training like some other trackers.

Thus, ByteTRACK allows us to easily replace detectors without having to redesign the architecture of the entire code.

The following is a simplified code snippet. In your own project, you need to use the starting code of ByteTRACK.

In the ByteTRACK project, every object on the frame has a unique ID.

This is the result of the model:

ByteTRACK single Frame Results

As you can see, each player is assigned a unique number and is tracked between frames.

Of course, detection and tracking are just the beginning.

With these, we can move to the next level!

For example, it is now possible to quickly analyze the course of an action to understand how the ball moves between players, calculate the distance a player has traveled, or locate the areas of the field where they appear most often.

Netizen: It would be a pity not to gamble

Some netizens said: Since you have so much information, it would be a pity not to gamble. It's a loss.

In this regard, some excited netizens said: "This is too dreamy! I have always wanted to make such a thing and sell it to football gamblers. People."

"You can show the passing status and preparation status of each offensive player every second. You can even conduct a 360-degree view of players who have never been passed. Performance evaluation, this is much more accurate than the estimated odds!"

Some netizens also said: Dream is a real-time mini-map for football matches.

Some serious netizens asked: "Is this script only applicable to a fixed camera angle, such as from one side covering the entire ground? Can it be used in other camera angles, such as focusing on 3-4 players?"

The owner of this post replied: "I think we can add use cases from other angles. If you use the data and retrain the model, it should perform just as well."

And the netizen said: "I feel that training data from another angle will not work. My idea is, Continue running the script while filming at full camera angle, but note the position of the ball and players around it, and tag the data on any other feeds."

The above is the detailed content of Stay up late tonight for Messi and Modric! This CV model allows you to win by guessing the ball. For more information, please follow other related articles on the PHP Chinese website!