Since the release of ChatGPT and Stable Diffusion, various related open source projects have flourished, which is really overwhelming.
Today, we focus on selecting a few high-quality open source projects to share with you, which will be of great help to our daily work, study and life.
This is an open source project from Microsoft. In just over a week, it has gained 23.6k stars.
Simply summarize it, it is a multi-modal question and answer system.
Supports AI painting, language question and answer, and picture question and answer, integrating the three recent hot spots in the AI industry.
Effect display:
The system implementation framework is as follows:
This is an open source project that “makes miracles with great effort”, integrating the research results of many parties: BLIP, CLIP, ChatGPT, pix2pix, inpainting, vqa et al.
To put it bluntly, it is to teach you how to use these projects to build a multi-modal question and answer system. This system architecture is of great reference value.
Project address:
https://github.com/microsoft/visual-chatgpt
This It is an open source project corresponding to a 2023 CVPR paper.
Just open sourced, fresh and hot~
The function is: based on a picture and a piece of audio, synthesize a video of the face speaking this voice.
Combined with ChatGPT, AIGC, and audio-to-text conversion, virtual two-dimensional or three-dimensional images can be "live".
In addition, the project has also made a plug-in for stable diffusion webui, which can be used directly in stable diffusion.
#The generated image can be directly matched with a piece of audio to generate a synthesized video.
Project address:
https://github.com/winfredy/sadtalker
Text can be edited Generate pictures? Can the video be edited?
FateZero: I can!
The left picture is the original picture, the right picture is the generated effect, the input text is:
Add Pokémon anime style:
Add Ink painting style:
# In addition to the style migration of the video, it also supports modification of the content inside.
For example: squirrel eats carrot, becomes, rabbit eats eggplant.
This project is also based on sd, which is a step closer to generating videos with one click.
Project address:
https://github.com/chenyangqiqi/fatezero
arXiv I believe everyone As we all know, the most popular paper hosting website currently has scientists and researchers from all over the world.
In order to improve the efficiency of arXiv users reading papers, someone has open sourced ChatPaper, an open source tool that uses ChatGPT to summarize arXiv papers.
The motivation of the developer, he told it like this:
In short, the project can be based on User keywords download the latest papers on arXiv, and use the powerful summary ability of ChatGPT3.5 API to condense them into a fixed format with less text and easy to read.
At the same time, the project supports individuals to deploy it themselves, or go directly to Hugge Face to experience it.
Project address:
https://github.com/kaixindelele/ChatPaper
https://huggingface.co/spaces/wangrongsheng/ChatPaper
There are too many companies all in ChatGPT recently, and various related open source projects are also emerging in endlessly.
Hope these projects can be helpful to you.