DeepMind機器人打乒乓球，正手、反手溜到飛起，全勝人類初學者

但可能打不過公園裡的老大爺？

巴黎奧運會正在如火如荼地進行中，乒乓球項目備受關注。同時，機器人打乒乓球也取得了新突破。

剛剛，DeepMind 提出了第一個在競技乒乓球比賽中達到人類業餘選手水平的學習型機器人智能體。

論文地址：https://arxiv.org/pdf/2408.03906

DeepMind 這個機器人打乒乓球什麼程度呢？大概和人類業餘選手不相上下：

正手反手都會：

對手採用多種打法，機器人也能招架得住：

不過，比賽激烈程度似乎不如公園老闆對戰。

對機器人來說，乒乓球運動需要掌握複雜的低階技能和策略性玩法，需要長期訓練。 DeepMind 認為戰略上次優但可以熟練地執行低階技能可能是更好的選擇。這使乒乓球與國際象棋、圍棋等純粹的戰略遊戲區分開來。

因此，乒乓球是提升機器人能力的一個有價值的基準，包括高速運動、即時精確和戰略決策、系統設計以及與人類對手直接競爭。

對於這一點，Google DeepMind 首席科學家稱讚道：「乒乓球機器人將有助於我們解決高速控制和感知問題。」

該研究進行了29 場機器人與人類的乒乓球比賽，其中機器人獲勝45% (13/29)。所有人類選手都是機器人未見過的玩家，從初學者到錦標賽選手能力不等。

雖然該機器人輸掉了所有與最高級別玩家的比賽，但它贏得了100% 的與初學者的比賽，在與中級選手的對戰中贏得了55% 的比賽，展現出人類業餘選手的水平。

總的來說，研究的貢獻包括：

1. 提出一個分層和模組化的策略架構，其中包括：

2. 低階控制器及其詳細的技能描述器，這些描述器對智能體的能力進行建模並有助於彌合模擬與真實的差距；

3. 選擇低階技能的高階控制器。

4. 實現零樣本模擬到真實的技術，包括定義基於現實世界的任務分佈的迭代方法，並定義自動課程（automatic curriculum）。

5. 即時適應未見過的對手。

方法介紹

該智能體由一個低階技能庫和一個高階控制器組成。低階技能庫專注於乒乓球的某個特定方面，例如正手上旋球、反手瞄準或正手發球。除了包含訓練策略，研究還在線上下和線上收集和儲存有關每個低階技能的優勢、劣勢和限制的資訊。而負責協調低階技能的高階控制器會根據當前遊戲統計、技能描述選擇最佳技能。

此外，該研究還收集了少量的人類和人類對打的比賽數據，作為初始任務條件的種子，數據集包括位置、速度和旋轉的資訊。然後使用強化學習在模擬環境中訓練智能體， 並採用一些現有技術，將策略無縫部署到真實硬體中。 

該智能體與人類一起對打以產生更多訓練數據，隨著機器人的持續學習，遊戲標準變得越來越複雜，以此讓智能體學習越來越複雜的動作。這種混合的「模擬 - 現實」循環創建了一個自動教學，使機器人的技能隨著時間的推移而不斷提高。 

Layered control

Layered control mainly includes the following parts:

• Table tennis playing style: The high-level controller (HLC, high-level controller) first decides which playing style to use (forehand or Backhand);

• Adjustment: Maintain each HLC's preference (H value) online based on statistics from matches against opponents;

• Select the most effective skill: HLC pairs shortlisted players based on adjusted H value Sampling by LLC;

• Updates: H-values ​​and opponent statistics are updated until the end of the game.

Results

The researchers compared the agent with 29 table tennis players of different levels, including beginners, intermediate, advanced and advanced + skills. Human players played three games against the robot according to standard table tennis rules, but the rules were slightly modified because the robot was unable to serve.

Facing all opponents, the robot won 45% of matches and 46% of games. Broken down by skill level, the bot won all its matches against beginners, lost all its matches against Advanced and Advanced+ players, and won 55% of its matches against Intermediate players. This shows that the agent reaches the level of an intermediate human player in table tennis rounds.

The reason why robots cannot beat advanced players is due to physical and technical limitations, including reaction speed, camera sensing capabilities, rotation processing, etc., which are difficult to accurately model in a simulation environment.

Sparring with robots is also very attractive

Research participants said that they enjoyed playing with robots very much and gave the robots high ratings in terms of "interesting" and "attractive" . They also unanimously expressed that they were "very willing" to fight the robot again. During free time, they played with the robot for an average of 4 minutes and 06 seconds over 5 minutes.

The robot is not good at backspin

The participant with the best skills mentioned that the robot is not good at handling backspin. To test this observation, the researchers plotted the robot's landing rate against the ball's spin, and the results showed that the robot's landing rate dropped significantly as it faced more backspin balls. This flaw is partly caused by the robot trying to avoid colliding with the table when handling low balls, and secondly by the fact that it is really difficult to determine the ball's spin in real time.

^{Reference link:}

^{https://sites.google.com/view/competitive-robot-table-tennis/home?utm_source&utm_medium&utm_campaign&utm_content&pli=1}

以上是DeepMind機器人打乒乓球，正手、反手溜到飛起，全勝人類初學者的詳細內容。更多資訊請關注PHP中文網其他相關文章！