Stunning! ! ! Tesla end-to-end demonstration video analysis

A user posted a video of Tesla FSD v12 on the Internet, and someone moved it to site B:

https://www.bilibili.com/video/BV1Z6421M797www.bilibili .com/video/BV1Z6421M797

This time it happens to be a purely visually complex scene: it rains and there is water on the ground, and various patterns are reflected on the water, which may produce some strange visual effects. Tesla did not hold an AI Day event last year, allegedly because competitors often provoked their slides, so it simply canceled it. In the absence of details, watching the video provides a glimpse into some of the end-to-end features. Next, let’s analyze some of the interesting points.

01:57, mistakenly detected that the car door was open, and took a large detour:

It’s not a big problem here, there is a relatively large space on the left, so It doesn’t matter if you go around a little longer.

02:09, misdetection of occ resulted in almost stopping:

The pedestrians have left and we can start moving forward. However, there was a lot of standing water on the ground, reflecting the image of the object, which could lead to false detections, so we stopped and waited for a while before moving on.

04:40, the close-range cut-in vehicle was missed

#The very close-range reversing cut-in vehicle on the left side was missed. But planning does not seem to give the intention of starting. This reflects a major advantage of end-to-end: the results of upstream errors do not necessarily lead to wrong driving behaviors. We will see more similar examples later.

05:37 False detection of occ

This may also be caused by the accumulation of water on the ground. This result was accepted end-to-end, and left and right Turn the steering wheel randomly, sometimes to the left and sometimes to the right.

05:48, misdetection of occ at close distances on the left and right

There are false detections of occ at very close positions on the left and right. If you still write according to the rules, it may be possible. You have to report it for takeover (not necessarily, after all, it is not on the driving track). Here, the two OCCs are ignored from end to end and continue driving.

06:57, False detection of a pedestrian at close range directly in front

This is really awesome, a pedestrian appears on the face, all rules-based The control system will definitely issue a sudden brake alarm at this time, but the end-to-end model does not recognize the upstream results and continues to drive as usual.

14 points: Wandering around in a private parking lot and unable to get out

#This may be a problem with the BEV's insufficient sensing distance, and it has not been found for a long time. Exit and circle around a parking lot. .

The other clips are all about driving on the main road. On the main road, the performance of FSD v12 is very smooth and there is no big problem. Especially at night, the detection of lane lines is also very stable, but I think it is a big problem. Some manufacturers can also achieve this level, so I won’t mention them here.

Looking at the section of the parking lot alone, if you don’t look at the upstream results, except for the misdetection of the front OCC that caused the steering wheel to turn left and right, the trajectory of FSD v12 is still relatively smooth, and even if there are errors, it is not stuck. living situation. In such a scene with pedestrians, irregularly moving obstacles (trolleys), and water on the ground, the performance is indeed okay.

Tesla still uses multi-tasking end-to-end with intermediate module supervision, so the front end can still display the results of obj det and occ. However, the end-to-end regulation does not necessarily accept the upstream results. A missed detection at a close range may not necessarily result in a start and a crash, and a misdetection at a close range may not necessarily cause the vehicle to stop. All results are input into the PNC for comprehensive judgment. . This is indeed an interesting point. What is certain is that Musk is not lying. This is indeed the performance of an end-to-end system.