Elephant P turns around and works right out of the box! HKU, NTU, Tsinghua University, etc. are the first to open source the 'replica' version of DragGAN

Do you remember the DragGAN released a few days ago?

That’s right, it’s the tool that allows you to retouch pictures in just two seconds.

#The photo you took has a bad expression? build! Face shape not thin enough? build! Is your face facing the camera at the wrong angle? build!

Maybe, the ancient PS joke "Let the elephant turn around", It may be coming true

As soon as this AI photo editing tool demonstration video was released, it became an instant hit at home and abroad.

Many netizens said, "PS no longer exists."

In just a few days, the unofficial implementation of DragGAN has become available for trial use. This function has been integrated into InternGPT. The interface looks like this↓

##Experience address: https:/ /igpt.opengvlab.com/

Unexpectedly, as soon as the demo entrance opened, it was immediately crowded.

Judging from the official demonstration video, the reproduced DragGAN effect is amazing.

Grin

However, judging from the output results, even if the forehead is selected, the hair will grow in equal proportions in all places, and the final result is a bit like the Monkey King.

Face Turn

##Face rotation is also a very practical function, and the completed part is very natural .

In addition to small-scale photo editing, InternGPT itself has many other eye-catching operations that can be performed.

Remove obscured objects

Click on the part of the picture you want to operate on, Just enter "remove" in the prompt.

Image generation

This function is more interesting. First upload an image, enter prompt to let DragGAN Split it, and then enter a prompt to generate the desired picture.

Exposed black feet? (No)

You can also edit videos with one click using prompt.

Interactive visual question and answer

Even after recognizing the information on the picture You can also query directly online.

Interactive image generation

You can easily doodle with just one click Become a beautiful picture.

Anyway, after reading these functions, the editor was really shocked. All functions highlight two features: "fool-like operation and ultimate utility."

Who can not love this?

After seeing so many cool features, what exactly is InternGPT?

InternGPT (iGPT for short)/InternChat (iChat for short) is a visual interaction system driven by pointing language. Users can interact with ChatGPT by clicking, dragging and drawing.

Different from existing interaction systems that rely on pure language, by integrating pointing instructions, iGPT significantly improves the communication efficiency between users and chatbots, as well as the ability of chatbots to perform vision-centric tasks. accuracy, especially in complex visual scenes.

Paper address: https://arxiv.org/pdf/2305.05662.pdf

The following figure shows the overall architecture of InternGPT.

We can see that this GPT can process not only images and videos, but also voice and text.

For image or video input, InternGPT will use SAM (image segmentation model), OCR (image recognition model), etc. to process it.

After identifying a geographical location, object or line, there is a whole toolbox for further processing, all of which are familiar tools.

For example, BLIP (audio), Stable Diffusion (image), Pix2Pix (image translation), etc.

Similarly, for text or voice input, InternGPT will call GPT-4, LLaMA and other models or tools for processing, and there will also be a whole toolbox later.

The overall architecture of InternGPT

Usage tips

The whole process is also very convenient during use.

After the user successfully uploads the image, he can send the following message to have a multi-modal related conversation with iGPT:

"what is it in the image?" or "what is the background color of image?".

Similarly, the user can also You can interactively operate, edit or generate pictures, as follows:

· Click anywhere on the picture, and then press the Pick button to preview the divided area. You can also press the OCR button to identify all words present at a specific location;

· To remove a masked area from an image, send the following message:

“remove the masked region”

· To replace the masked object with another object in the image, you can send the following message:

“replace the masked region with {your prompt}”

· If you want to generate a new image, you can send the following message:

“generate a new image based on its segmentation describing {your prompt}”

· To create a new image by doodling, press Whiteboard and draw on the whiteboard. After the drawing is completed, you need to press the save button and send the following message:

“generate a new image based on this scribble describing {your prompt}”

Netizen comments

That shocking DragGAN There is now an unofficial version. The official version will be released in June, this is just a preview of the future.

DragGAN has been integrated into InternGPT, and it came out so quickly. It is a picture editing tool.

The above is the detailed content of Elephant P turns around and works right out of the box! HKU, NTU, Tsinghua University, etc. are the first to open source the 'replica' version of DragGAN. For more information, please follow other related articles on the PHP Chinese website!