Tian Yuandong: Some thoughts on GPT-4

OpenAI released GPT-4 yesterday, and it was really good after trying it out.

#In terms of story generation, which I am mainly concerned about, compared with the stories generated by ChatGPT, GPT-4 has better writing style, more details, and is better at being vivid and concrete. The plot also began to take some interesting turns. Of course, some of the problems of ChatGPT, such as the lack of overall layout and story planning ability, the characters are not prominent, the narrative development is boring, the continuation paragraph will quickly end in a mediocre ending, etc., still exist on GPT-4.

Judging from its performance report, compared with GRE verbal which is close to full score, GRE writing has only 4 points and Advanced Placement English writing has only 2 points ( 3 points passing), and there is no progress from GPT-3 to 4. I don’t know if it is related to these problems.

##But anyway, GPT-4 It can already help us with many daily tasks. Only ten years have passed since deep learning began to explode in 2013. We have such powerful tools, and their evolution speed can be said to be beyond imagination.

#In 2017, why do some math students dislike and even despise Deep Learning? There were some bold guesses in this answer, such as "I fumbled around in the long night, and the sun has not yet appeared." Now it seems that these predictions are generally correct, but I didn't expect that this day would come so soon.

#Once a technology becomes a universal tool like calculus, the huge changes that will follow can be expected.

#If we take a longer view, what will happen in the future?

#I also had some thoughts around 2020 and put them into my own science fiction novels. In general, if this trend develops to the extreme, and even the manufacturing of physical products can be fully automated using local materials, then the final result will be "what you think is what you get": as long as you can conceive of a thing, it will It will appear in front of you at a very low cost.

As long as most repetitive labor can be copied unlimitedly by machines, its value will drop to almost zero and become as ubiquitous as water and air. public goods.

#Under such conditions, "uniqueness" will become a value and meaning that everyone cherishes. Perhaps in the future, the main factor in judging talents will no longer be completing a degree or passing an exam, but looking at personal experience and achievements.

"Doing better than others with the same goal" is unique, and "going in a direction that no one has ever gone before" is also unique . This not only includes academic achievements recognized by peers, such as solving problems that no one can solve, creating and exploring a new academic direction, but also includes unique experiences, insights, ways of thinking, and even three views, such as a person who uses crutches to hike A programmer who has explored Antarctica, what views, insights and insights will he/she have about the world? How will a history student who once made a homemade rocket to go to the moon lose control and was miraculously rescued tell this experience in his autobiography?

These rights to speak closely combined with personal characteristics will allow everyone to occupy a unique ecological niche in human society and become "human beings" The ultimate sea-fixing needle. Even if consciousness can be uploaded and digital twins can be realized in the future, this unique personality cannot be copied - as long as the exact same individual becomes two and begins to receive different external signal stimulations, their internal states will change. Depending on the update, the trajectory will start to diverge, making it unique again.

In general, everyone proves their choices through their own "Tao", and accumulates their own unique characteristics because of past choices. It is impossible to judge in advance who is better than whom, because calculating this judgment function itself requires individuals to practice their own path. And when countless people who insist on their own directions gather together, various wonderful ideas will continue to emerge. Some of them have been proven correct in practice, which will affect others and attract followers. And a diverse and inclusive world that includes all these individuals will be full of motivation for progress.

I’m going too far.

#For researchers, what does the emergence of GPT-4 mean?

We have two extreme choices. One is to close the door, not listen or see, and use the old thinking inertia to conceive the research route and continue Publish the next article; or you can completely abandon your previous research ideas, tear up all the textbooks, and believe that computing power is king, and big models and big data are everything.

#I think both of these attitudes are undesirable. The paradigm of research needs to change. It is no longer about discovering specific knowledge, but about studying the mechanism of model learning and seeking the origin of its learning ability. For example, through massive data training, existing models can automatically learn many concepts.

But how does the model learn this concept? Why do large models learn faster and better? How is it discovered from the data and captured by the model? And is it constantly reinforced during training? How are advanced concepts formed? How do existing models capture these concepts, and with what mechanism? I think these may be truly meaningful basic research directions in the long run, and they are also the final outlet for truly transforming the metaphysics of "artificial intelligence" into science.

Of course, these should be said to be viewed on a scale of several decades. To do these studies, you need a lot of basic knowledge, sufficient patience, and good Accept new things that are beyond your scope of knowledge and integrate them into your original workflow and research ideas.

This can be regarded as my unique "Dinghai Shenzhen" in research, and I hope to continue on.

The above is the detailed content of Tian Yuandong: Some thoughts on GPT-4. For more information, please follow other related articles on the PHP Chinese website!