HaoMo Zhixing & Tsinghua AIR self-driving open class Dr. Pan Xing revealed HaoMo's self-driving AI practice on the spot

As AI large models emerge from deep learning algorithms, they are becoming the hottest new technology paradigm in the current AI field. Autonomous driving technology also has the possibility to evolve from the modular stage to end-to-end autonomous driving due to the introduction of large model technology. AI large models are reshaping the technical route of autonomous driving.

On September 27, the high-quality open course on autonomous driving jointly organized by HaoMo Zhixing and Tsinghua University Intelligent Industry Research Institute (AIR) concluded successfully. This open class focuses on the current leading AI algorithms for autonomous driving, combined with Haomo’s specific practices, bringing an end-to-end autonomous driving technology feast to autonomous driving practitioners, industry partners and media friends.

This course is the third in a series of open courses on autonomous driving. The first and second courses previously provided a basic introduction to the autonomous driving knowledge system from the perspective of macro industry and technical principles. In the third issue, Dr. Zhan Xianyuan, assistant researcher/assistant professor at Tsinghua AIR, explained the characteristics and current progress of end-to-end autonomous driving AI algorithms from the perspective of decision optimization, while Dr. Pan Xing, technical director of HaoMo Zhixing, fully explained from the data closed-loop system How the AI ​​large model algorithm learns and optimizes in massive data, and how it demonstrates its amazing capabilities in practice.

In the sharing titled "End-to-End Autonomous Driving from the Perspective of Decision Optimization", Dr. Zhan Xianyuan started from the concept of end-to-end and combined with the 30-year development history of the autonomous driving industry to tell everyone about end-to-end autonomous driving. The strategic learning method in the algorithm, combined with the scientific research cooperation practice of Tsinghua AIR and Hao Mo, uses the Al algorithm to gain insight into the development context of the industry and make summary and trend judgments on the development of the industry.

Dr. Zhan Xianyuan pointed out that end-to-end, simply put, is to integrate all architectures and different modules into a complete whole, conduct training directly from input to output, and transmit learning signals forward from the decision-making point. The advantage of the original modularity is that each module is disassembled very cleanly, the modeling goal of each module is very clear, and the interpretability is very good. However, under the modular architecture, each module has its own system for design and optimization. Combining multiple modules together will inevitably lead to accumulation of errors. The advantages of end-to-end lies in the following three points. First of all, the entire end-to-end model can be regarded as a single very large model, so the structure is very simple. All goals are optimization and learning around the ultimate goal of decision-making. The goals are unified at the optimization level. Secondly, it is end-to-end learning from input to final decision output, which can easily implement pure data-driven learning that relies on massive data. Third, because many models are trained in the same system end-to-end, the backbones of different module models can be shared, thereby reducing computational overhead.

Dr. Zhan Xianyuan introduced that all end-to-end driving models can be regarded as a large decision-making model. Training such a model requires the use of decision-making optimization algorithms. This involves imitation learning and reinforcement learning. Imitation learning is a mapping directly trained from data using supervised learning methods; reinforcement learning is not simply imitating data. It provides the possibility of transcending the data itself and can be optimized through continuous learning. Find a better decision-making model than the available data.

In Dr. Zhan Xianyuan’s view, the early end-to-end autonomous driving models were all small decision-making models, but today, the end-to-end systems implemented in the industry are all huge models. The online interaction paradigm has gradually extended to completely offline learning. As models become stronger and better, security becomes better and better, and there are slowly some improvements and transitions at the generalization level. In addition, Dr. Zhan Xianyuan also gave a detailed introduction to the cooperation between Tsinghua AIR and Haimou in imitation learning and offline reinforcement learning, and said that these algorithms will gradually be applied to Haimou’s autonomous driving scenario practice.

After Dr. Zhan Xianyuan’s sharing, Dr. Pan Xing took “Hai Mo’s Road to Autonomous Driving AI” as the theme and explained the importance of AI algorithms from an industrial perspective through Hao Mo’s specific practice. Dr. Pershing said that as an artificial intelligence technology company dedicated to autonomous driving, Haimou users have driven assisted driving for more than 80 million kilometers. Urban NOH is also in the process of generalization and iteration, and is expected to achieve mass production next year.

Pershing said that with the increase in data scale, improvement in algorithm capabilities, and applications under the trend of big models, big data, and big computing power, the current industry is about to enter the data-driven era of autonomous driving 3.0. As autonomous driving products are moving from high-speed scenarios to urban scenarios, the construction of autonomous driving data intelligence systems is the core infrastructure. In order to achieve a closed data loop, companies such as Tesla, Haimo and many domestic companies are also building their own cloud AI capabilities and supercomputing centers to achieve better results through greater computing power and larger-scale data processing capabilities. autonomous driving capabilities.

Currently, Haomo has built its own data intelligence system MANA, and at the beginning of this year built the largest intelligent computing center in China's autonomous driving industry - MANA OASIS Snow Lake·Oasis. Based on MANA OASIS, Haomo released the industry's first autonomous driving generative large model DriveGPT Xuehu·Hairuo in April this year. "As a basic large model, Haimo uses DriveGPT to build further AI capabilities, including data management retrieval, automatic annotation, AIGC simulation data synthesis, etc. Based on these data capabilities and services, we further improve the performance of various modules and algorithms on the vehicle end. capabilities, and ultimately achieve a better autonomous driving product.”

Dr. Pan Xing pointed out that data intelligence is the core of the entire autonomous driving iteration. In this process, massive data assets must be accumulated. Through large AI models, these data assets can be better managed. At the same time, computing power is inevitably needed after data is available. The stable and continuous operation of the intelligent computing center also provides a steady stream of power for the iteration of large models and the improvement of autonomous driving.

Dr. Pan Xing said, "With data and computing power, the current car-cloud linkage and joint training methods can effectively improve the effect of car-side algorithms through large models." For example, by using DriveGPT, in the tool chain It can very effectively reduce the cost of the entire labeling and improve the efficiency of labeling. At the same time, DriveGPT can also use large models to directly support the improvement of the capabilities of small models on the car side, and better transfer the large model capabilities of the cloud to the models on the car side.

Dr. Pershing also said, “How to obtain more realistic simulation data efficiently, large models can play a very important role.” Through the use of large models, texture, depth, semantics and other information can be learned very effectively. Through the effective representation of large models, the data can be edited. For example, vehicle obstacles that are not in the original video can be pasted, edited, rotated at will, and put into the video through DriveGPT, thereby obtaining New simulation synthetic data. In addition to applications in the field of perception, large models also play a great role in intelligent driving decision-making and planning. DriveGPT uses user data of human driving to continuously iterate and learn to achieve better driving behavior and decision-making. .

At the same time, Haimo DriveGPT can not only help complete trajectory prediction and image synthesis, but also has the ability to make intelligent decisions. "DriveGPT has the ability to input a video to predict future trajectories and answer questions in the driving decision-making process, and can give explainable decisions." These capabilities make Weimo believe that with the advent of end-to-end autonomous driving, macro decision-making and Micro-behavior, learning and understanding together through models will become a more effective means. Dr. Pershing revealed that next, Haimou will connect the two models of perception and cognition more deeply end-to-end so that they can be integrated into one.

HaoMo Zhixing and Tsinghua University Intelligent Industry Research Institute (AIR) organized a high-quality public course on autonomous driving in 4 phases, and this course is the third. In the two previous courses, lecturers from Tsinghua AIR and Hao Mo Zhixing have introduced the development of autonomous driving technologies such as single-vehicle intelligent autonomous driving, vehicle-road collaborative autonomous driving and high-level intelligent road construction to nearly a hundred industry media people. Everyone explained the basic principles of autonomous driving AI technology and the current application trends of large models in autonomous driving. In this high-quality open course on autonomous driving, Haimo and Tsinghua AIR shared more in-depth AI algorithms and principles of autonomous driving AI systems with observers in the autonomous driving industry, and received active questions and exchanges from online and offline guests.

Facing the self-driving stars and the sea, only action can truly achieve the future goal. Through the high-quality open courses on autonomous driving, Haimo and Tsinghua AIR have joined hands with senior media people in the industry to harvest the latest research results and practical experience on autonomous driving A algorithms. They have joined hands across mountains and seas to share the wisdom of AI knowledge and contribute to the autonomous driving industry. Precious technical consensus and knowledge accumulation.

The above is the detailed content of HaoMo Zhixing & Tsinghua AIR self-driving open class Dr. Pan Xing revealed HaoMo's self-driving AI practice on the spot. For more information, please follow other related articles on the PHP Chinese website!