AI watched 70,000 hours of 'Minecraft' videos to learn advanced human skills, netizen: It's so painful

What level can AI reach by watching 70,000 hours of "Minecraft" videos?

Take the "diamond pickaxe" as an example. It takes an advanced human player 20 minutes of rapid clicking and about 24,000 actions to complete it.

And the current AI can already hold it easily.

哐哐Find various materials and perform various synthesis step by step:

This is the "Minecraft" AI from OpenAI, which is known as the strongest - —MineDojo.

It is also the world's first AI that can create "diamond tools".

Not only that, but building "stone pickaxes" and "simple shelters" are not a problem:

Of course, in "Minecraft" Other routine operations are also easily handled by MineDojo.

For example, swimming, hunting, pillar jumping, etc.:

As for why OpenAI wants AI to learn these skills, researcher Bowen Baker said:

This is largely because we are simulating human behavior when surfing the Internet.

How is MineDojo made?

As we just mentioned, the "way to refine" MineDojo is to watch videos.

These video contents are posted by human players on YouTube to show how they play "Minecraft".

Then after watching 70,000 hours of video, the AI ​​learned how to perform various tasks in the game.

This method is generally called imitation learning, which is to train the neural network to learn by observing human behavior.

Although there have been many related studies before, there are still some problems that need to be solved.

"Tagging" is one of them.

The traditional way is to put a label on every action: if you do this, this will happen, if you do that, that will happen.

But the conceivable consequence of this method is that the workload is too large, resulting in less data that can be used for training.

Therefore, OpenAI researchers took a different approach and came up with a different research idea - Video Pre-Training (VPT):

The core idea of ​​​​this method is to train another neural network specifically to handle the tedious "labeling" work.

To this end, the researchers first found a group of players and asked them to play "Minecraft" first. Of course, while having fun, they also had to record the number of keyboard and mouse clicks.

In this way, the researchers first obtained some 2000 hours of labeled data.

On this basis, they began to train a model to match keyboard and mouse movements and on-screen results -

For example, under what circumstances will clicking the mouse cause the game to character wielding an axe.

After training this model, 70,000 hours of unlabeled videos will be introduced; with its support, a huge and usable data set will be born.

The next step is to return to the previous idea of ​​imitation learning and use these new data to train AI.

Although imitation learning can be said to be a branch of reinforcement learning, OpenAI researchers found that AI trained by VPT can complete tasks that cannot be accomplished by reinforcement learning alone. .

Such as making wooden planks and turning them into a table (requires approximately 970 consecutive actions).

Not only that, the researchers also found that if imitation learning and reinforcement learning are combined, the effect will be the best.

Expanding from this research on "Minecraft", OpenAI researchers also said:

Our AI can also perform other tasks, such as browsing websites with the mouse and booking flights. Or shop online.

"Minecraft" has become a test field for AI technology

In fact, the highlight of OpenAI's research this time is that, excluding the VPT method itself, the two major elements of its research-"Minecraft" ” and videos have also become the focus of heated discussion.

A major feature of the game "Minecraft" is its openness. Players can make many unexpected masterpieces in this virtual world.

Different from the past game environments where reinforcement learning trained AI, most of them ended with "win or loss" as the result, but often the ability of AI later trained may exceed this "limitation".

But there is no such thing as "win or lose" in "Minecraft". AI can play its full role here. Therefore, OpenAI researchers said:

"Minecraft" is a good experimental field for training AI.

And this also won the recognition of NeurIPS - MineDojo won an award at this year's top conference.

As for the second hot topic of this study, "video," as Sony Executive Director Peter Stone said:

Video is a training resource with great potential.

But it seems that OpenAI researchers are not satisfied with this result. They believe that collecting 1 million hours of "Minecraft" videos will make their AI even better.

Of course, this research also attracted a lot of attention from netizens, and there were also some interesting discussions:

People want AI to be conscious, but only after they become conscious do they realize that they need to be It's tiring to watch videos for so long.

Paper address: https://openai.com/blog/vpt/

Reference link:

[1]https ://www.reddit.com/r/technology/comments/z58fmi/a_bot_that_watched_70000_hours_of_minecraft_could/

[2]https://www.youtube.com/watch?v=Z2FsxrRmDPQ[3]https:// www.youtube.com/watch?v=fJn9B64Znrk​

The above is the detailed content of AI watched 70,000 hours of 'Minecraft' videos to learn advanced human skills, netizen: It's so painful. For more information, please follow other related articles on the PHP Chinese website!