Will the autonomous driving annotation industry be subverted by the world model in 2024?

1. Problems faced by data annotation (especially based on BEV tasks)

With the rise of BEV transformer-based tasks, it brings about the need for data Dependencies are becoming heavier and heavier, and annotation based on BEV tasks is becoming more and more important. At present, whether it is 2D-3D joint obstacle annotation, lane lines based on reconstructed point cloud clips, or Occpuancy task annotation, it is still too expensive (compared to 2D annotation tasks, it is much more expensive). Of course, there are also many semi-automatic or automated annotation studies based on large models in the industry. On the other hand, the data collection cycle for autonomous driving is too long and involves a series of data compliance issues. For example, if you want to collect a scene where a flatbed truck crosses the camera, or a scene where the number of lanes in the city changes from more to less, or from less to more, you need a collection staff to specially build such a scene.

Will 2.24 be the singularity moment for the world model?

The concept of world model covers a wide range and can be considered as sensor simulation. At Tesla AI Day, I witnessed for the first time the disruptive impact of simulation technology on data annotation.

Figure 1: Tesla’s automated annotation effectFigure 2 4D reconstruction effect

When I first saw this news , I was very surprised, as if I had discovered a Tesla electric car that subverted the tradition. As more and more researchers invest in this field, we are seeing many excellent research results. UniSim's autonomous driving simulation system has functions such as replay, dynamic object behavior control, and free-view rendering. These functions are very desirable for every model researcher who is training.

You can also simulate lidar.

For details, see: https://zhuanlan.zhihu.com/p/636695025. There are more similar studies in this direction.

NeuRAD: Neural Rendering for Autonomous Driving

DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes Most of the methods are related to Nerf, and the entire pipeline is relatively heavy. There is another direction, a research direction based on diffusion. Good research has also been done.

BEVControl: Accurately Controlling Street-view Elements withMulti-perspective Consistency via BEV Sketch Layout

##

Technology The development is too fast, and the threshold for sensor simulation is lowering. It is possible that some disruptive products will appear in the autonomous driving labeling industry in 2024!