Google AI rising star switches to Pika: video generation Lumiere, serves as founding scientist

Video generation is progressing in full swing, and Pika welcomes a great general-

Google researcher Omer Bar-Tal, who serves as Pikafounding scientist.

A month ago, Google released the video generation model Lumiere as a co-author, and the effect was amazing.

At that time, netizens said: Google joins the video generation battle, and there is another good show to watch.

Some people in the industry, including Stability AI CEO and former Google colleagues, sent their blessings.

One work by Lumiere, just graduated with a master's degree

Omer Bar-Tal, graduated from Tel Aviv with a bachelor's degree in 2021 Department of Mathematics and Computer Science at the university, and then went to the Weizmann Institute of Science to pursue a master's degree in computer science, mainly focusing on research in the field of image and video synthesis.

The results of his papers have been accepted by top conferences many times, such as Text2LIVE (ECCV 2022 Oral), MultiDiffusion (ICML 2023), and TokenFlow (ICLR 2024).

Taking TokenFlow as an example, they proposed a framework to implement text-driven video editing based on the diffusion model, supporting video editing tasks without any training or fine-tuning.

Before joining Pika this time, he worked as a Student Researcher at Google Research for 9 months. Finally, after 7 months of research, he launched the project as a co-author. Lumiere. At this time, the master's degree should not have been completed yet.

Lumiere’s innovation lies in the proposed spatio-temporal U-Net (STU-Net) architecture: down-sampling and up-sampling the video in both spatial and temporal dimensions. , the compressed spatio-temporal representation of the video is obtained in the middle layer of the network.

After learning 30 million videos, Lumiere can support multiple functions such as recording videos, video editing and repair, image conversion to video, and video stylization.

At that time, Jeff Dean praised: The multi-modal video generation revolution is happening.

Will Smith Eating Noodles (Pika Version)

After the official announcement of joining Pika, officials and investors also sent their blessings.

A few days ago, Yilun Du, a Chinese researcher at Pika, published an article, but he should have just graduated from MIT with a PhD (or may not have graduated yet), the thesis unit is still MIT.

In addition, the official website is still recruiting people.