Tesla's Optimus humanoid robot works in the factory, is skilled in disassembling batteries, self-correcting, and can go even further

Tesla humanoid robot has unlocked new skills!

Yesterday, Tesla Optimus officially released a new demo video, showing the latest progress of the second-generation Optimus humanoid robot.

#This time, Optimus started working in the factory, learned how to assemble batteries at the Tesla battery factory, and walked faster than before Further and more stable.

Let’s take a first look at Optimus’ latest skills and training details.

Optimus’ end-to-end neural network is now trained to accurately package battery cells at Tesla factories.

Running in real time on the robot's FSD computer, relying solely on 2D cameras, hand touch and force sensors . Optimus uses its legs to maintain balance while the network drives its entire upper body.

The insertion process requires very precise movements, and the fault tolerance rate is very low. The neural network will automatically locate the next free slot. Optimus is also able to autonomously recover from failures.

Optimus’ training data is collected via human teleoperation and scaled for a variety of tasks.

Optimus is being tested at the factory and human intervention rates continue to decrease.

Of course, Optimus can now regularly walk around the office, not only faster but also farther and farther.

In this regard, Tesla Optimus (Tesla Bot) engineer Milan Kovac explained more details of Optimus’ training.

Over the past few months, Tesla’s amazing manufacturing team has built more robots to conduct research and collect labor Smart data.

The team trained and deployed a neural network that allowed Optimus to begin performing useful tasks, such as picking up battery cells from a conveyor belt and inserting them into pallets with precision.

This neural network operates completely end-to-end, meaning that Optimus only consumes video from the robot's 2D camera and data from the onboard proprioceptors and directly produces joint control sequences.

Optimus runs entirely on the robot’s embedded FSD computer, powered by an onboard battery. This design allows a single neural network to perform multiple tasks by adding more diverse data to the training process.

Although Optimus is not perfect yet and is still a bit slow, we are seeing higher and higher success rates and fewer and fewer mistakes. Tesla also trained Optimus to recover from failure cases and saw spontaneous corrections occur.

Tesla has deployed several Optimus humanoid robots in a factory, where they are tested daily on real workstations and continuously improved.

The team is conducting further work to make Optimus move faster and handle rougher terrain without sacrificing its human nature. The team will also focus on repeatability, training neural networks to handle dynamic calibration and small differences between robots. There will be more updates soon.

In addition, Jim Fan, senior research scientist and head of AI Agents at Nvidia, gave high praise to the skills demonstrated in Optimus’s new video.

# He believes that the video gives us a glimpse into the real situation of human data collection, which is Optimus’s biggest advantage. What does it take to build such a pipeline? Optimus does the following:

Optimus's hand is one of the world's top five-finger dexterous robot hands. It has tactile perception and has 11 degrees of freedom (DOF). Many competing products have only 6-7 DOF, and are highly robust and can withstand a large number of object interactions without the need for continuous maintenance.

Remote operation software: We can see human operators wearing VR glasses and gloves. Setting up the software to enable real-time streaming of first-person video and precise control of the output while maintaining extremely low latency is pretty remarkable. Humans are very sensitive to the smallest delay between their own movements and that of a robot. Optimus has a fluid full-body controller that can perform human poses in real time.

Large-scale fleet: You need more than one robot to collect data in parallel, a trained human working multiple shifts per day (preferably around the clock), and an on-call Maintenance team to ensure the robot is always busy. That’s a lot of operational complexity that academic research labs don’t even consider.

Task and Environment: It is equally important to figure out what you want to operate remotely. Currently, most of this work is demo-driven: collect data on tasks you want to put into social media videos. But to solve the problem of general-purpose robots, we need to carefully consider the distribution of tasks and environments. From the 43- to 51-second portion of the video, we can see factory and home settings, such as moving batteries, processing laundry, and sorting everyday items into shelves.

This is an open research question: If you only had the budget to collect training data for 1000 tasks, what would you choose to maximize skill transfer and generalization?

Conclusion: Teleoperation is necessary, but not sufficient, to solve the problem of humanoid robots. It's fundamentally unscalable.

In the comment area, Musk previewed the latest goal for the Optimus humanoid robot hand, which will have 22 degrees of freedom (DoF) later this year. This is even more exciting.

Previously, according to Musk’s revelation on the earnings call, Tesla’s Optimus humanoid robot may have a “useful factory” by the end of this year. "Task" capabilities, and plans to take the lead in using it in its own factory before the end of this year. At the same time, Tesla is likely to sell Optimus before the end of 2025.

"Evolutionary History" of Tesla Humanoid Robot

The Tesla humanoid robot can be traced back to "Tesla" in August 2021. At the "AI Day" event, Musk first revealed his idea of ​​building a humanoid robot (Tesla Bot).

This humanoid robot is about 1.72 meters tall and weighs about 112 pounds. It has a well-proportioned body and a facial screen. It can move at a speed of about 8 kilometers/hour.

In October 2022, at this year’s “Tesla AI Day”, the Optimus humanoid robot made its debut. At that time, it could not perform dance or other tricks. The robot consumes 500W of power when working, weighs 73 kilograms, and has 27 degrees of freedom for hand movement.

However, in the Demo demonstration, Optimus showed pictures of it moving boxes, watering flowers, and working in the Tesla Gigafactory.

#In May 2023, after more than half a year of dormancy, Tesla Optimus finally gained the ability to walk smoothly and grab objects.

# In September 2023, Tesla Optimus evolved again and can classify objects autonomously. The highlight is the complete end-to-end training of the neural network : Input video, output control. In his spare time, he can also do yoga for fun.

In December 2023, the second generation of Optimus arrived, making people really watch the movie "Public Enemy" What it looks like inside.

# With the release of a new video, the speed of Tesla’s humanoid robot progress is once again stunning.