MoDa community launches AI video generation tool Live Portait, which can make photos speak with one click

Magic Community has launched an AI video generation tool called Live Portrait, which can make the characters in the photo speak with one-click operation

Alibaba Cloud has launched a digital human video generation tool called Live Portrait. Users only need to upload a photo and a text or voice to generate a talking digital human video. This tool can be used in many scenarios such as live video broadcasts, chat robots, and corporate marketing. Currently, this tool is open for experience in the Magic Community Creation Space

With the popularity of self-conversation large models and AI painting models, the research community is gradually pushing the research on generative AI into more modalities, among which AI video generation technology has attracted much attention. This technology can convert information such as text or audio into facial movement information to generate animated photos with character images, effectively lowering the threshold for video shooting and production

Alibaba Cloud’s latest Live Portait tool combines the motion module and the generation module. This tool uses Alibaba Cloud's self-developed mouth shape prediction algorithm, which greatly improves the accuracy of mouth shape generation and is significantly improved compared to traditional methods. In the training stage, explicit control of posture is added, so that the generated video can show any action without the need for a baseboard video, thus greatly improving the realism of digital human speech. In addition, through active eye control technology, Live Portait can add natural movement to the eyeballs, making the generated results closer to real-life effects. According to reports, Live Portait related technologies have been included in top international AI conferences such as CVPR and ICCV

According to information from the Magic Community, Live Portait provides two methods for users to choose from after uploading photos, namely text-driven and audio-driven. In text-driven mode, users can choose from 28 different voices, including Mandarin, English, Cantonese, and children's voices. In addition, Live Portait also provides lightweight model selection to help users generate videos faster

Zhang Bang, head of the tool’s algorithm, said: “Live Portait integrates a number of innovative technologies independently developed by the team, including the ability to generate realistic facial animations using a single picture, breaking through the limitations of traditional adversarial generation networks. With the continuous evolution of technology, image-generated videos have broad application prospects and are expected to become an important tool for enterprises to improve production efficiency and reduce costs."

It is understood that the team’s research directions include digital humans, 3D model AI generation, high-fidelity rendering and natural human-computer interaction. It has published more than 50 papers at top international conferences

The above is the detailed content of MoDa community launches AI video generation tool Live Portait, which can make photos speak with one click. For more information, please follow other related articles on the PHP Chinese website!