|
|
@@ -1,69 +1,71 @@
|
|
|
-English | [简体中文](README_CN.md)
|
|
|
+简体中文 | [English](README.md)
|
|
|
|
|
|
# Image Matting
|
|
|
|
|
|
-## Contents
|
|
|
-* [Introduction](#Introduction)
|
|
|
-* [Update Notes](#Update-Notes)
|
|
|
-* [Community](#Community)
|
|
|
-* [Models](#Models)
|
|
|
-* [Tutorials](#Tutorials)
|
|
|
-* [Acknowledgement](#Acknowledgement)
|
|
|
-* [Citation](#Citation)
|
|
|
+## 模型下载
|
|
|
+- [通用目标抠图](https://paddleseg.bj.bcebos.com/matting/models/deploy/ppmatting-hrnet_w48-composition.zip)
|
|
|
+- [人物抠图](https://paddleseg.bj.bcebos.com/matting/models/ppmattingv2-stdc1-human_512.pdparams)
|
|
|
|
|
|
+## 目录
|
|
|
+* [简介](#简介)
|
|
|
+* [更新动态](#更新动态)
|
|
|
+* [技术交流](#技术交流)
|
|
|
+* [模型库](#模型库)
|
|
|
+* [使用教程](#使用教程)
|
|
|
+* [社区贡献](#社区贡献)
|
|
|
+* [学术引用](#学术引用)
|
|
|
|
|
|
-## Introduction
|
|
|
|
|
|
-Image Matting is the technique of extracting foreground from an image by calculating its color and transparency.
|
|
|
-It is widely used in the film industry to replace background, image composition, and visual effects.
|
|
|
-Each pixel in the image will have a value that represents its foreground transparency, called Alpha.
|
|
|
-The set of all Alpha values in an image is called Alpha Matte.
|
|
|
-The part of the image covered by the mask can be extracted to complete foreground separation.
|
|
|
+## 简介
|
|
|
+
|
|
|
+Image Matting（精细化分割/影像去背/抠图）是指借由计算前景的颜色和透明度，将前景从影像中撷取出来的技术，可用于替换背景、影像合成、视觉特效，在电影工业中被广泛地使用。
|
|
|
+影像中的每个像素会有代表其前景透明度的值，称作阿法值（Alpha），一张影像中所有阿法值的集合称作阿法遮罩（Alpha Matte），将影像被遮罩所涵盖的部分取出即可完成前景的分离。
|
|
|
|
|
|
|
|
|
<p align="center">
|
|
|
<img src="https://user-images.githubusercontent.com/30919197/179751613-d26f2261-7bcf-4066-a0a4-4c818e7065f0.gif" width="100%" height="100%">
|
|
|
</p>
|
|
|
|
|
|
-## Update Notes
|
|
|
+## 更新动态
|
|
|
* 2022.11
|
|
|
- * **Release self developed lite matting SOTA model PP-MattingV2**. Compared with MODNet, the inference speed of PP-MattingV2 is increased by 44.6%, and the average error is decreased by 17.91%.
|
|
|
- * Adjust the document structure and improve the model zoo information.
|
|
|
- * [FastDeploy](https://github.com/PaddlePaddle/FastDeploy) support PP-MattingV2, PP-Matting, PP-HumanMatting and MODNet models.
|
|
|
+ * **开源自研轻量级抠图SOTA模型PP-MattingV2**。对比MODNet, PP-MattingV2推理速度提升44.6%， 误差平均相对减小17.91%。
|
|
|
+ * 调整文档结构，完善模型库信息。
|
|
|
+ * [FastDeploy](https://github.com/PaddlePaddle/FastDeploy)部署支持PP-MattingV2, PP-Matting, PP-HumanMatting和MODNet模型。
|
|
|
* 2022.07
|
|
|
- * Release PP-Matting code. Add ClosedFormMatting, KNNMatting, FastMatting, LearningBaseMatting and RandomWalksMatting traditional machine learning algorithms.
|
|
|
- Add GCA model.
|
|
|
- * upport to specify metrics for evaluation. Support to specify metrics for evaluation.
|
|
|
+ * 开源PP-Matting代码；新增ClosedFormMatting、KNNMatting、FastMatting、LearningBaseMatting和RandomWalksMatting传统机器学习算法；新增GCA模型。
|
|
|
+ * 完善目录结构；支持指定指标进行评估。
|
|
|
* 2022.04
|
|
|
- * **Release self developed high accuracy matting SOTA model PP-Matting**. Add PP-HumanMatting high-resolution human matting model.
|
|
|
- * Add Grad, Conn evaluation metrics. Add foreground evaluation funciton, which use [ML](https://arxiv.org/pdf/2006.14970.pdf) algorithm to evaluate foreground when prediction or background replacement.
|
|
|
- * Add GradientLoss and LaplacianLoss. Add RandomSharpen, RandomSharpen, RandomReJpeg, RSSN data augmentation strategies.
|
|
|
-
|
|
|
+ * **开源自研高精度抠图SOTA模型PP-Matting**；新增PP-HumanMatting高分辨人像抠图模型。
|
|
|
+ * 新增Grad、Conn评估指标；新增前景评估功能，利用[ML](https://arxiv.org/pdf/2006.14970.pdf)算法在预测和背景替换时进行前景评估。
|
|
|
+ * 新增GradientLoss和LaplacianLoss；新增RandomSharpen、RandomSharpen、RandomReJpeg、RSSN数据增强策略。
|
|
|
* 2021.11
|
|
|
- * **Matting Project is released**, which Realizes image matting function.
|
|
|
- * Support Matting models: DIM, MODNet. Support model export and python deployment. Support background replacement function. Support human matting deployment in Android.
|
|
|
-
|
|
|
-## Community
|
|
|
-
|
|
|
-* If you have any questions, suggestions and feature requests, please create an issues in [GitHub Issues](https://github.com/PaddlePaddle/PaddleSeg/issues).
|
|
|
-* Welcome to scan the following QR code and join paddleseg wechat group to communicate with us.
|
|
|
+ * **Matting项目开源**, 实现图像抠图功能。
|
|
|
+ * 支持Matting模型：DIM， MODNet；支持模型导出及Python部署；支持背景替换功能；支持人像抠图Android部署。
|
|
|
+
|
|
|
+## 技术交流
|
|
|
+
|
|
|
+* 如果大家有使用问题和功能建议, 可以通过[GitHub Issues](https://github.com/PaddlePaddle/PaddleSeg/issues)提issue。
|
|
|
+* **欢迎加入PaddleSeg的微信用户群👫**（扫码填写简单问卷即可入群），大家可以和值班同学、各界大佬直接进行交流，还可以**领取30G重磅学习大礼包🎁**
|
|
|
+ * 🔥 获取深度学习视频教程、图像分割论文合集
|
|
|
+ * 🔥 获取PaddleSeg的历次直播视频，最新发版信息和直播动态
|
|
|
+ * 🔥 获取PaddleSeg自建的人像分割数据集，整理的开源数据集
|
|
|
+ * 🔥 获取PaddleSeg在垂类场景的预训练模型和应用合集，涵盖人像分割、交互式分割等等
|
|
|
+ * 🔥 获取PaddleSeg的全流程产业实操范例，包括质检缺陷分割、抠图Matting、道路分割等等
|
|
|
<div align="center">
|
|
|
<img src="https://user-images.githubusercontent.com/30883834/213601179-0813a896-11e1-4514-b612-d145e068ba86.jpeg" width = "200" />
|
|
|
</div>
|
|
|
|
|
|
-## Models
|
|
|
-
|
|
|
-For the widely application scenario -- human matting, we have trained and open source the ** high-quality human matting models**.
|
|
|
-According the actual application scenario, you can directly deploy or finetune.
|
|
|
+## 模型库
|
|
|
|
|
|
-The model zoo includes our self developded high accuracy model PP-Matting and lite model PP-MattingV2.
|
|
|
-- PP-Matting is a high accuracy matting model developded by PaddleSeg, which realizes high-resolution image matting under semantic guidance by the design of Guidance Flow.
|
|
|
- For high accuracy, this model is recommended. Two pre-trained models are opened source with 512 and 1024 resolution level.
|
|
|
+针对高频应用场景 —— 人像抠图，我们训练并开源了**高质量人像抠图模型库**。根据实际应用场景，大家可以直接部署应用，也支持进行微调训练。
|
|
|
|
|
|
-- PP-MattingV2 is a lite matting SOTA model developed by PaddleSeg. It extracts high-level semantc informating by double-pyramid pool and spatial attention,
|
|
|
- and uses multi-level feature fusion mechanism for both semantic and detail prediciton.
|
|
|
+模型库中包括我们自研的高精度PP-Matting模型和轻量级PP-MattingV2模型。
|
|
|
+- PP-Matting是PaddleSeg自研的高精度抠图模型，通过引导流设计实现语义引导下高分辨率图像抠图。追求更高精度，推荐使用该模型。
|
|
|
+ 且该模型提供了512和1024两个分辨率级别的预训练模型。
|
|
|
+- PP-MattingV2是PaddleSeg自研的轻量级抠图SOTA模型，通过双层金字塔池化及空间注意力提取高级语义信息，并利用多级特征融合机制兼顾语义和细节的预测。
|
|
|
+ 对比MODNet模型推理速度提升44.6%， 误差平均相对减小17.91%。追求更高速度，推荐使用该模型。
|
|
|
|
|
|
-| Model | SAD | MSE | Grad | Conn |Params(M) | FLOPs(G) | FPS | Config File | Checkpoint | Inference Model |
|
|
|
+| 模型 | SAD | MSE | Grad | Conn |Params(M) | FLOPs(G) | FPS | Config File | Checkpoint | Inference Model |
|
|
|
| - | - | -| - | - | - | - | -| - | - | - |
|
|
|
| PP-MattingV2-512 |40.59|0.0038|33.86|38.90| 8.95 | 7.51 | 98.89 |[cfg](../configs/ppmattingv2/ppmattingv2-stdc1-human_512.yml)| [model](https://paddleseg.bj.bcebos.com/matting/models/ppmattingv2-stdc1-human_512.pdparams) | [model inference](https://paddleseg.bj.bcebos.com/matting/models/deploy/ppmattingv2-stdc1-human_512.zip) |
|
|
|
| PP-Matting-512 |31.56|0.0022|31.80|30.13| 24.5 | 91.28 | 28.9 |[cfg](../configs/ppmatting/ppmatting-hrnet_w18-human_512.yml)| [model](https://paddleseg.bj.bcebos.com/matting/models/ppmatting-hrnet_w18-human_512.pdparams) | [model inference](https://paddleseg.bj.bcebos.com/matting/models/deploy/ppmatting-hrnet_w18-human_512.zip) |
|
|
|
@@ -74,28 +76,27 @@ The model zoo includes our self developded high accuracy model PP-Matting and li
|
|
|
| MODNet-HRNet_W18 |35.55|0.0035|31.73|34.07| 10.2 | 28.5 | 62.6 |[cfg](../configs/modnet/modnet-hrnet_w18.yml)| [model](https://paddleseg.bj.bcebos.com/matting/models/modnet-hrnet_w18.pdparams) | [model inference](https://paddleseg.bj.bcebos.com/matting/models/deploy/modnet-hrnet_w18.zip) |
|
|
|
| DIM-VGG16 |32.31|0.0233|28.89|31.45| 28.4 | 175.5| 30.4 |[cfg](../configs/dim/dim-vgg16.yml)| [model](https://paddleseg.bj.bcebos.com/matting/models/dim-vgg16.pdparams) | [model inference](https://paddleseg.bj.bcebos.com/matting/models/deploy/dim-vgg16.zip) |
|
|
|
|
|
|
-**Note**：
|
|
|
-* The dataset for metrics is composed of PPM-100 and human part of AIM-500, with a total of 195 images, which named [PPM-AIM-195](https://paddleseg.bj.bcebos.com/matting/datasets/PPM-AIM-195.zip).
|
|
|
-* The model default input size is (512, 512) while calculating FLOPs and FPS and the GPU is Tesla V100 32G. FPS is calculated base on Paddle Inference.
|
|
|
-* DIM is a trimap-base matting method, which metrics are calculated in transition area.
|
|
|
- If no trimap image is provided, the area 0<alpha<255 is used as the transition area after dilation erosion with a radius of 25 pixels.
|
|
|
-
|
|
|
-## Tutorials
|
|
|
-* [Online experience](docs/online_demo_en.md)
|
|
|
-* [Quick start](docs/quick_start_en.md)
|
|
|
-* [Full development](docs/full_develop_en.md)
|
|
|
-* [Human matting android deployment](deploy/human_matting_android_demo/README.md)
|
|
|
-* [Human matting .NET deployment](https://gitee.com/raoyutian/PaddleSegSharp)
|
|
|
-* [Dataset preparation](docs/data_prepare_en.md)
|
|
|
-* AI Studio tutorials
|
|
|
- * [The Matting tutorial of PaddleSeg](https://aistudio.baidu.com/aistudio/projectdetail/3876411?contributionType=1)
|
|
|
- * [The image matting tutorial of PP-Matting](https://aistudio.baidu.com/aistudio/projectdetail/5002963?contributionType=1)
|
|
|
-
|
|
|
-## Acknowledgement
|
|
|
-* Thanks [Qian bin](https://github.com/qianbin1989228) for their contributons.
|
|
|
-* Thanks for the algorithm support of [GFM](https://arxiv.org/abs/2010.16188).
|
|
|
-
|
|
|
-## Citation
|
|
|
+**注意**：
|
|
|
+* 指标计算数据集为PPM-100和AIM-500中的人像部分共同组成，共195张，[PPM-AIM-195](https://paddleseg.bj.bcebos.com/matting/datasets/PPM-AIM-195.zip)。
|
|
|
+* FLOPs和FPS计算默认模型输入大小为(512, 512), GPU为Tesla V100 32G。FPS基于Paddle Inference预测库进行计算。
|
|
|
+* DIM为trimap-based的抠图方法，指标只计算过度区域部分，对于没有提供trimap的情况下，默认将0<alpha<255的区域以25像素为半径进行膨胀腐蚀后作为过度区域。
|
|
|
+
|
|
|
+## 使用教程
|
|
|
+* [在线体验](docs/online_demo_cn.md)
|
|
|
+* [快速体验](docs/quick_start_cn.md)
|
|
|
+* [全流程开发](docs/full_develop_cn.md)
|
|
|
+* [人像抠图Android部署](deploy/human_matting_android_demo/README_CN.md)
|
|
|
+* [人像抠图.NET部署](https://gitee.com/raoyutian/PaddleSegSharp)
|
|
|
+* [数据集准备](docs/data_prepare_cn.md)
|
|
|
+* AI Studio第三方教程
|
|
|
+ * [PaddleSeg的Matting教程](https://aistudio.baidu.com/aistudio/projectdetail/3876411?contributionType=1)
|
|
|
+ * [PP-Matting图像抠图教程](https://aistudio.baidu.com/aistudio/projectdetail/5002963?contributionType=1)
|
|
|
+
|
|
|
+## 社区贡献
|
|
|
+* 感谢[钱彬(Qianbin)](https://github.com/qianbin1989228)等开发者的贡献。
|
|
|
+* 感谢Jizhizi Li等提出的[GFM](https://arxiv.org/abs/2010.16188) Matting框架助力PP-Matting的算法研发。
|
|
|
+
|
|
|
+## 学术引用
|
|
|
```
|
|
|
@article{chen2022pp,
|
|
|
title={PP-Matting: High-Accuracy Natural Image Matting},
|
|
|
@@ -104,3 +105,6 @@ The model zoo includes our self developded high accuracy model PP-Matting and li
|
|
|
year={2022}
|
|
|
}
|
|
|
```
|
|
|
+
|
|
|
+## 参考文档
|
|
|
+https://gitee.com/paddlepaddle/PaddleSeg/blob/release/2.8/Matting/docs/quick_start_cn.md
|
