Kaifu Lee: Large AI models are a historical opportunity that cannot be missed

"In the future, I think the most revolutionary AI2.0 application will be AI-First." Li Kaifu, chairman of Sinovation Ventures, said in his speech.

What is AI-First application? It refers to applications that cannot exist without the large model.

On May 28, Li Kaifu, Chairman and CEO of Sinovation Ventures and Dean of the Artificial Intelligence Engineering Institute of Sinovation Ventures, delivered a speech titled "New Opportunities from AI1.0 to AI2.0" at the 2023 Zhongguancun Forum.

In his speech, he gave his own opinions on those who had doubts about the development of large models, including the commercial value of large model development, the impact of AI2.0 on the future application ecology, and the prospects for the development of large models in China. .

He said that even an imperfect large model has a commercial value of tens of trillions of dollars. The future AI2.0 market is expected to be 10 times the size of the mobile Internet, accommodating giants, small and medium-sized enterprises, and startups at three levels. We look forward to forming an “innovation complex” where giants, small, medium and micro innovative enterprises can develop together.

The following is the full text of Kai-Fu Lee’s speech (with abridgements)

I am very happy to have this opportunity to look at the recent AI 2.0 from the perspective of our investment institution. I may be the person who knows the most about AI among the investment institutions, and I may also know the best about investing in the field of AI. I am very excited because in the past two years, the emergence of AI2.0 and large-scale models has made me read almost all relevant papers in the past five years. The investment and technology teams of Sinovation Ventures are also studying this field. For this reason, they even went to the United States to learn about some of the latest developments in OpenAI, Google, and Microsoft.

What I want to introduce today is not the companies invested by Sinovation Ventures or other actual businesses, but the recent many questions about AI2.0 large models. I hope I can give a relatively simple explanation that will be helpful to everyone.

01

Business value of large models

What is AI2.0? From the perspective of Sinovation Ventures, from the perspective of investment generating economic value, the development stages of AI are divided into AI1.0 and 2.0. Deep learning developed into AI1.0 after AlphaGo, and was subsequently widely used in various industries and created value.

Since 1989, the development of AI1.0 has begun. However, in recent years, some difficulties have been encountered that can be solved by large models. What are the bottlenecks encountered in AI1.0? When there were no large models at that time, if we wanted to apply AI in a field, we needed to collect, clean, and label data in that field, and then use it to adjust the model. The cost of the whole process was very expensive.

For big companies like Douyin, Alibaba or Baidu, there is no problem. They collect too much data and there are many opportunities to make money. But if you are a bank or insurance company Companies or factories, then there will be huge problems in implementing AI applications, and the cost will be unbearable.

The advantage of large models is that they can be fully trained once and then applied through transfer learning or fine-tuning. In the AI 1.0 era, each application is an isolated island. However, with the emergence of large models, massive data can be used to train a basic large model at once, and then this model can be used to adjust the required applications, thus reducing the cost a lot. For example, speaker products like "Xiao Ai" may be confused once they step out of the field that "Xiao Ai" understands. You will find that the smart speaker is not that smart in fact. It can only do some things such as playing music. , simple things like checking the weather, and there are many things that cannot be done. But when you put something like ChatGPT in, its knowledge reserve is activated. As long as you adjust its conversation mode, it can become a very good conversation robot.

A very perfect large-scale model has considerable value even in business, and can even reach a scale of trillions. It depends on how humans cooperate with it, which is the so-called Human in the loop.

Reporters use AI to help them write, or lawyers use AI to help them with litigation. In the end, as long as humans are still responsible for the article or litigation, and AI only does the preliminary writing, there will be no problem. Considering that people have learned that artificial intelligence can make mistakes, we only allow it to initially provide draft manuscripts instead of directly serving users to protect users from being harmed by misinformation.

AI gives full play to the advantages of its massive data base. For example, it can write a new summary based on reading 1,000 articles, and write a summary based on reading the past 10,000 historical lawsuits. These high-quality summaries Reporters or lawyers can get useful information and let people check whether AI has made any mistakes.

Also, in some fields, people don’t actually care about AI making mistakes. For example, in entertainment applications, it is harmless to be a hero in a game and have a longer beard or a shorter one, or it may say the wrong sentence. It doesn't matter, it's all made up in the game anyway. In fact, many application fields can tolerate these imperfect large models. Based on the analysis of this fact, AI has great potential in productivity applications. Of course, some fields are very critical and cannot tolerate mistakes, such as news search, government websites, or medical and education-related fields. These fields are very difficult to do. In the future, the problem of large model mistakes needs to be solved.

02

AI2.0 rewrites the application ecosystem

We can completely imagine that today’s Word, PowerPoint, Excel, Photoshop and other applications will all be rewritten using AI large models, and after rewriting, their user experience will change, and even the business model will change. There are also some areas where the cost of making a mistake is not too high, such as push advertising. If an advertisement is pushed wrong, it will be harmless. Today we have been pushed a lot of advertisements by mistake, and the advertisements we receive when we turn on TV and websites are not targeted, but AI can make advertisements more targeted, although occasionally it will make mistakes.

So just in the above areas, I think it is an opportunity worth tens of trillions of dollars. Of course, we have to continue to work hard to reduce the frequency of its nonsense. Here is a complete set of methods, from pre-training data to training alignment, to subsequent processing, as well as some early warning and temporary quick correction methods. These combinations Together, I believe we can do it.

Productivity is the greatest opportunity. AI2.0 has roughly three ecosystems. We usually talk about the bottom layer as the basic model. What we just talked about is the top application layer, such as helping you write manuscripts, write lawsuits, draw pictures, cut out pictures, etc. There is also a middle layer that provides various tools to optimize models and implement transfer learning to help large models be applied more efficiently. The middle layer has two parts. The first one is to expand outward from the basic model layer. For example, if the model is so large, can it be reduced when inference is needed, and a large model can be turned into a small model for a certain field? , or the issue just mentioned about reducing the frequency of nonsense.

The other is the adjustment from the application layer down. For example, when we want to rewrite a Photoshop, you can say one sentence and the picture will come out, but you may need to go further and say, "I want to change the color of the rainbow." , or the male-female ratio of the audience inside can be adjusted. This requires intelligent cutting of the large model and an understanding of some objects. These are actually not directly related to the large model itself, but without these functions, it is impossible to push Photoshop to someone who wants to draw pictures.

The middle layer is actually very important. What does the middle layer make us think of? For example, the purpose of the middle layer provided by Windows, Android, and Apple is very simple - to minimize the cost of application development. Only in this way can the number of applications increase and create a virtuous cycle in which users bring more applications.

In my previous speech, I mentioned the arrival of the AI 2.0 era - a platform plus application model. Once this platform technology is mastered, it will transform every field. We can clearly feel that, for example, when making a game, the creation of all your characters, including backgrounds, props, clothes, and even all codes, will eventually be written using AI, so it is very likely that some children will use it in the community in the future. Write down the game you want to play, and everyone can use words to introduce the game to each other, and you can play it in a few seconds.

The e-commerce and advertising mentioned just now are also an example. We can tailor advertisements and pictures for each person according to his needs, cognition, education level and shopping habits. This will maximize the Increase his desire to buy. Of course, there will be regulatory issues here. What if what you write is false or hurts users? This still requires legal supervision, but I hope everyone can understand these two examples. The big model is really not just a question and answer engine. It has changed the ecology of all APPs. It will rewrite every ecosystem we use today. .

03

To be AI-First in the era of large models

Big models reform not only artificial intelligence, but also bring about some huge platform-like gaps. Among all applications, AI-First will be the most important application. What is AI-First? It’s just that if this application doesn’t have AI, it won’t work. For example, some of the Mobile-First applications we use now, such as Meituan, Didi, and Douyin, develop applications based on the premise that mobile phones are on people all day long. Without the mobile phone, these applications cannot be used.

These companies have made full use of the functions brought by mobile phones, developed new applications adapted to mobile phones, and obtained our geographical location, so that we can use them to complete activities such as taxi hailing and takeout. Then other companies, such as Sina, NetEase, Douban, etc., they also did very well in the mobile Internet era, but they only moved their PC applications, so they did not get the same explosion. If you wanted to start a business or invest in the mobile Internet back then, you had to choose apps that had to be mobile. So if you want to start a business in the field of artificial intelligence today, you have to make apps that have to have artificial intelligence.

Simply put, AI-First means that without large models, the application becomes completely useless. This kind of application is what we need to do today. It will be the darling of this era in the future. Its entire user experience may be more about learning to communicate with us in human language, rather than forcing us to learn computer language. .

Of course we all know that there are still many challenges, including issues such as false information and privacy protection. Emphasizing that strengthening supervision is necessary, we also need more technology to solve these problems. It is definitely not enough to use supervision or technology alone, and the two methods should be combined.

Recently, there have been some controversial voices in the market, such as “You can use large overseas open source models to make a Chinese version of OpenAI”, “There is no need to make large models, small models are enough”, “Large models are expensive and time-consuming” People, only giants have admission tickets" "There are too many large-model startups in China" and so on. In my opinion, open source is very important. China's technology will definitely need open source in the future, because there are still entrepreneurs in universities who do not have open source and it is difficult to get the power to start.

But we must not believe what some people say on the Internet. I took an open source model, such as GPT-4, to train, and suddenly found that the model was as good as GPT-4, so the large model has no value, and you don’t need to do it. Yes, this is absolutely wrong.

Because first, the open source model itself may have limitations. When you train a large model, it requires a lot of GPUs and the cost is relatively high. The open source model basically sets the ceiling of your model, and then you do the alignment adjustment and learning work inside. The improvements that these tasks bring to the model are determined by your ceiling. If your goal from the beginning is a ceiling like GPT-4, then it will be impossible to make a model larger than that. Second, many people use GPT-4 to train their open source models, but we really cannot guarantee that GPT-4 will continue to be open for use in the future. Fine-tuning a model trained overseas for use domestically is dangerous. Because the culture, habits, laws and regulations at home and abroad are different, if you take a set of models trained in the United States and debug them in China, do you think those frameworks can solve domestic problems?

So I still believe that it is necessary to open source large models. Of course, there will not be 50 large model companies in the future. This will shrink to a relatively small number, just like there were about 10 American search engines when they first started. , and later through mergers and acquisitions, the remaining five or six companies have developed very well. For example, Google was the last one to come up, but it has developed into the first one. So I think there will be more specific companies, whether they will be mainly large companies or small companies. It is not possible to make a conclusion too early. Everyone still has a chance. .

Today, especially when we are still catching up in this field, we should still encourage various development models, because it is difficult for us to know who can do it.

There are three steps in the evolution of large models. The first step is medium-sized large models. Most Chinese participants are at this step. The second step is to cross the threshold of "emergence" and become mainstream large models. At present, some large model companies in China It has reached a data scale of 60 billion and is roughly in the second stage. However, the quality of China's data is not high enough. If we want to enter the third stage, both data quality and data scale are important; the third step is to become a leading big data player. model company. Currently, only two foreign companies have reached the third step. Their model data scale and data quality are very good, and they also have human feedback reinforcement learning, which can be linked to many downstream applications.

The point I want to talk about is in terms of models. In particular, I think there is one thing in OpenAI’s GPT-4 that everyone has not noticed, which is its model expansion (scale up) function. This function is said to be able to use one thousandth Or 1/10,000 times to predict whether the training of a model will be successful.

When we train a large model for a month, if we make a mistake, we will waste thousands of GPUs. Then this set of scale up functions quantifies the possibility of successful training to a certain extent and can reduce waste. . However, we currently don’t know how OpenAI operates this scale up. We can only try to do this from some papers they issued. Even if we lack a GPU, it is still necessary to understand how to maximize the use of our GPU.

Model size is not the only determining factor in the development of large models, data quality is more important. In the AI 1.0 era, when we train various language models, the more data, the better. A little mistake is not harmful, but the experience we have gained from training large models is that both data quality and quantity are important, but compared to Quality cannot be sacrificed, and I think this needs the help and promotion of the country.

The quality of Internet data in the United States is higher than that in China. For example, if my family has any health problems, I must go to WebMD or Cleveland Clinic to find them, but there are no similar websites in China. China currently does not have a public welfare data collection platform, so we still need the power of the national government to promote the collection of high-quality data and narrow the gap in data quality with foreign countries. The next gap between China and the United States may be reflected in this data quality.

04

AI2.0 Outlook

I believe that the most revolutionary AI2.0 application in the future will be AI-First. In the end, those who stand out are the pioneers who dare to fully invest in new technologies. In the past, humans needed to learn computer languages, but in the future it may be computers learning our languages, which will greatly save our time. We just need to tell AI what we want to do and it will do it for us. For example, I can tell the smart assistant that tomorrow is my wife’s birthday and I need flowers, cakes, and gifts, and it will help me get them all, which saves me a lot of time. We are currently recruiting assistants to help us with these tasks to save time. In the future, these tasks can be completed by AI assistants.

In the future, I also believe that the big model will not just be a chat tool, but will slowly exceed user expectations and develop into an intelligent productivity tool. When this day finally arrives, we will find that the current business model will change and the App Store will no longer exist.

For example, when I tell the AI that I want to buy cakes and flowers for my wife’s birthday, it does not need to go to the e-commerce website. It can place an order directly with the warehouse, so it will subvert the existing business model and bring more customers. economic opportunity. The AI applications we see now are basically in the virtual world, but in the future AI will move to the physical world. We have a concept called Embodied AI, which means that if you use massive videos as training data, it is possible for the robot to understand your needs. If you tell it to bring some potato chips, it will know that it needs to open the drawer, take the plate, pour it out, etc. Of course, it may be difficult to see them enter practical applications in the next three to five years, but academic and industrial The combination makes it all seem not so far-fetched.

So the large AI model is a historical opportunity that China cannot miss. This is the largest platform revolution in history. It is 10 times greater than the changes brought about by windows and Android. It will rewrite all applications and reconstruct human work, allowing creative people to better focus on research work, amplifying their ingenuity by 10 times or more, and at the same time, many repetitive tasks will be replaced.

Although China started later than the United States, we still have great potential for development due to the huge application market and the strong connectivity of all aspects of our economy. The Chinese government performs better than Western countries in allocating resources and arranging work, and can successfully guide more Chinese people into positions suitable for them. China also has a huge talent advantage. China has a very large number of AI engineers and AI scientists. Perhaps the top ones are still in the United States, but many young scientists in China are also very powerful. However, we still have a challenge. In the past, our computing power was not as good as that of the United States, and we had less experience in large models. However, I believe that with the joint efforts of the government, large enterprises, and investment companies, we will soon be able to overcome this problem. question.

Andreessen Horowitz, a well-known American investment institution, has a prediction for this field: "The potential size of this market is difficult to grasp - it will be between all software and all human efforts." AI2. 0 The market is expected to be 10 times the size of the mobile Internet, and it is expected to form an "innovation complex" where giants, small, medium and micro innovative enterprises will develop together.

Source: China Entrepreneur Magazine

The above is the detailed content of Kaifu Lee: Large AI models are a historical opportunity that cannot be missed. For more information, please follow other related articles on the PHP Chinese website!