Hot on GitHub! This artifact is open source!

Recently, the world’s major new car manufacturing companies simply cannot be more popular! The editor is really jealous when he sees the soaring stock price. Anyone who knows the industry knows that major companies, led by Tesla, use computer vision as the technical base for autonomous driving. It is through image segmentation technology that cars can clearly distinguish where the road is and where the people are.

Do I still need to emphasize whether image segmentation is important? The open source suite I want to introduce to you today covers the industry’s most cutting-edge image segmentation algorithms and has outstanding results. This is PaddleSeg! ! OMG, what are you waiting for? ! Plate him! Plate him! Plate him!

At the scheduled global computer vision conference CVPR2021, PaddleSeg once again shined. Among them, the AutoNUE Challenge is a very influential event in the field of autonomous driving scene understanding in recent years, which tests the contestants' semantic segmentation algorithm capabilities in unstructured environments. The Baidu PaddleSeg team finally defeated the other participating teams and won the championship by ranking first in the three test indicators of Level 1, Level 2, and Level 3.

Anxious friends can go directly to the game details:

https://bj.bcebos.com/paddleseg/docs/autonue21_presentation_PaddleSeg .pdf

So what is PaddleSeg? The editor went to GitHub to find out the official explanation:

PaddleSeg is an end-to-end image segmentation development kit developed based on PaddleSeg, which covers a large number of high-quality segmentation models in different directions such as high precision and lightweight. . Through modular design, it helps developers complete the full-process image segmentation application from training to deployment. Let me tell you about the features and recent updates of PaddleSeg:

The portrait segmentation function has been newly upgraded and an ultra-lightweight model deployment solution on the web is provided;

Launched the refined segmentation solution PaddleSeg-Matting;

Open sourced the panoramic segmentation algorithm Panoptic-DeepLab, enriching model types;

Released intelligent annotation for interactive segmentation ToolsEISeg. Greatly improves labeling efficiency.

Web Video Conferencing

##Matting

Panorama Segmentation

Interactive Segmentation

provides an industrial-level deployment method. Now so many new features have been added. It can be said that PaddleSeg can meet the needs of developers in all dimensions in an all-round and three-dimensional way. I have to say loudly: ,

# Such a good product, why don’t you get on it quickly?

Getting on the bus address:

https://github.com/PaddlePaddle/PaddleSeg

Industrial grade Portrait segmentation scheme PPSeg

Portrait segmentation is a very common application in the field of image segmentation. In practical applications, portrait data sets come from various sources. The data may come from mobile phones, cameras, surveillance, etc., pictures The size may be landscape, portrait or square. There are various deployment scenarios, some applications are on the server side, some applications are on the mobile side, and some applications are on the web side. To this end, the PaddleSeg team has launched a portrait segmentation PPSeg model trained on large-scale portrait data to meet the needs of multiple usage scenarios on the server, mobile, and web (Paddle.js).

The PPSeg model has been widely used in the industry. Recently, "Baidu Video Conference" has also launched a virtual background function, allowing users to switch backgrounds during video conferences. Among them, the portrait background changing model adopts the ultra-lightweight model in the PPSeg series of models developed by the PaddleSeg team. Through Paddle.js, it is deployed on the web side and directly uses the computing power of the browser to perform image segmentation. The segmentation effect has been well received.

Detailed explanation of industrial-level solutions:

https://github.com/PaddlePaddle/PaddleSeg/tree/release/2.2/contrib/HumanSeg

Friends can also go to Baidu homepage to experience Baidu video conference and intuitively experience the portrait segmentation function provided by PaddleSeg and Paddle.js.

Refined Segmentation Solution PaddleSeg-Matting

With the development of segmentation technology, people’s requirements for refined segmentation are getting higher and higher. For example, in some film and television industries, green screens are commonly used to change the background of shooting, but if the goal is not to shoot in front of a green screen, can it still achieve a good background segmentation function?

The answer is: Yes!

The recent open source fine segmentation solution PaddleSeg-Matting by the PaddleSeg team solves this problem very well. The target hair is accurately segmented.

PaddleSeg achieves alpha prediction through the built-in trimap generation mechanism, which can complete the prediction without the input of any auxiliary information, greatly reducing labor costs. Reduce the number of network parameters by sharing encoder weights, and use the attention module in the decoder stage to implement trimap information flow guidance for alpha prediction. Then use the error map to extract the patch of the misestimated area, and refine it through the refinement sub-network to obtain the final alpha.

Interactive Segmentation Intelligent Annotation Tool

The industry has this saying about artificial intelligence: "How intelligent is deep learning? That’s how much labor there is.” This sentence directly expresses the pain in the hearts of deep learning practitioners. After all, the quality of the model plays a big factor, but the cost of data labeling gives many practitioners a headache.

For this reason, the PaddleSeg team has launched the interactive segmentation intelligent annotation software EISeg. So what exactly is interactive segmentation? Find out through the dynamic chart below.

It is not difficult to find that interactive segmentation realizes the comparison through a series of green points (positive points) and red points (negative points) For edge segmentation of target objects, the main application directions of interactive segmentation are image editing and semi-automatic annotation. It can be applied to scenarios such as refined annotation, cutout, and auxiliary image post-processing (such as PS).

The PaddleSeg team and PaddleCV-SIG members have launched the industry's first high-performance interactive segmentation tool EISeg based on the RITM algorithm. We support the entire process of training, prediction and interaction of the RITM model. The PaddleSeg interactive segmentation model not only supports training powerful general scene models from scratch, but also supports Finetune on specific scene data. We use Baidu's self-built portrait data set to finetune the model and obtain a portrait interactive segmentation model with fast prediction speed, high accuracy and few interaction points.

The software provides a variety of installation methods, supporting users to use pip and conda to install. In addition, an executable exe file is provided under windows. Double-click the .exe to run the program.

Panoptic-DeepLab

Panoramic segmentation is a new field that has emerged in the field of image segmentation in recent years. It was first proposed by FAIR and Heidelberg University in 2018.

What is panoramic segmentation?

The information of the image can be divided into thing and stuff, where thing represents countable objects, such as cars, animals, etc., and stuff represents uncountable objects, such as beaches, sky, etc. The semantic segmentation task does not pay attention to whether the image is stuff or thing, but only focuses on the semantic category to which each pixel belongs, so it cannot distinguish instance objects. Instance segmentation focuses on the segmentation of things, identifying things in the image, distinguishing different individual instances and corresponding semantic information, and the stuff area is uniformly represented as the background. Panoramic segmentation is a technology that combines semantic segmentation and instance segmentation. For thing, different individual instances and corresponding semantic information are identified. For stuff, the corresponding semantic information is identified.

Panoptic DeepLab achieves state-of-the-art performance for the first time in the form of bottom-up and single-shot algorithms. Compared with the top-down algorithm, Panoptic DeepLab achieves both accuracy and speed with a simple network structure, creating a pioneering A new direction for panoramic segmentation algorithms. The current top ranking of Cityscape panoramic segmentation is based on this algorithm.

PaddleSeg full view

• All-star algorithm lineup

20 The high-precision semantic segmentation algorithm is comprehensively ahead of similar frameworks, and the 50 pre-trained model has a new panoramic segmentation algorithm, enriching application scenarios. Provides a high-precision portrait segmentation algorithm HumanSeg to meet multi-end deployment.

• Full industry chain deployment

not only fully supports the development of dynamic graphics, but also can smoothly complete the conversion of dynamic graphics; it also The entire process and links such as data preprocessing, algorithm training and tuning, compression, and multi-terminal deployment are smoothly connected, which greatly improves the ease of use of user development and accelerates the speed of industrial application of algorithms. In particular, Paddle.js supports web-side deployment, giving more possibilities for web-side deployment.

What are you waiting for? ! Such a high-quality product developed with such care, why not hurry up and collect it in your Star collection!

Portal:

https://github.com/PaddlePaddle/PaddleSeg