Weakly Supervised Semantic Segmentation list

This repository contains lists of state-or-art weakly supervised semantic segmentation works. Papers and resources are listed below according to supervision types.

There are some personal views and notes, just ignore if not interested.

Last update 2019/2

• Paper list
  • instance
  • box
  • one-shot
  • others
• Resources

some unsupervised segment proposal methods and datasets here.

CVPR 2018 Tutorial : WSL web&ppt, Part1 ,Part2

Typical weak supervised segmentation problems

No	Supervision	Difficulty	Domain	Core issues
1	Bounding box	middle	annotated classes	transfer learning
2	One-shot segment	middle	similar objects	one-shot learning
3	Image/video label	hard	annotated classes	transfer learning
4	Others	n/a	n/a	n/a

1.Bounding box supervision

Instance semantic segmentation

• Learning to Segment Every Thing, CVPR 2018

  :Learning weight transfer from well-annotated subset, transfer class-specific weights(output layers) from detection and classification branch, based on Mask-RCNN

• Pseudo Mask Augmented Object Detection, CVPR 2018

  :State-of-art weakly supervised instance segmentation with bounding box annotation. EM optimizes pseudo mask and segmentation parameter like Boxsup. Graphcut on superpixel is employed to refine pseudo mask.

• Simple Does It: Weakly Supervised Instance and Semantic Segmentation, CVPR 2017 [web] [ref-code][supp]

  :Grabcut+(HED bounday) and MCG , train foreground segmentation network directly with generated mask semantic segmentaion, sensitive to env(quality) of training images. Check my implementation for pseudo mask generation which is similar to Grabcut+ MCG. But it can't match the performance discribed in paper sup. Opencv version Grabcut perform even worse.

• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation, ICCV 2015

  :Based on CRF refine, EM seems not work

• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation, ICCV 2015

  :Iteratively update parameters and region proposal labels, proposals are selected by network output masks

• Deepcut: Object segmentation from bounding box annotations using convolutional neural networks, TMI 2017

Arxiv paper

• Learning to Segment via Cut-and-Paste, Arxiv 1803

• Adversarial Learning for Semi-Supervised Semantic Segmentation, Arxiv1802, [code]

2.One-Shot segmentation supervision

DAVIS Challenge: http://davischallenge.org/

: Davis17/18(Semi-supervised Video segmentation task), Davis16 is video salient object segmentation without the first frame annotations.

• Fast and Accurate Online Video Object Segmentation via Tracking Parts, CVPR 2018(Spotlight) [code]

  :state-of-art, 82.4%/1.8s 77.9%/0.6s

• OSVOS: One-Shot Video Object Segmentation, CVPR 2017 [web][code]

  :milestone, fine-tuning parent network with the first frame mask, 79.8%/10s

3.Image/video label supervision

• Self-produced Guidance for Weakly-supervised Object Localization, ECCV 2018

• Convolutional Simplex Projection Network (CSPN) for Weakly Supervised Semantic Segmentation, BMVC 2018

• Weakly Supervised Instance Segmentation using Class Peak Response, CVPR 2018(Spotlight)

  :state-of-art practice for instance seg with only class label.

• Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Features, CVPR 2018

  :Superpixel-> RegionNet(RoI classfier)-> Saliency refine, iteratively update with PixelNet(FCN)

• Revisiting Dilated Convolution: A Simple Approach for Weakly- and SemiSupervised Semantic Segmentation, CVPR 2018(Spotlight)

• Weakly-Supervised Semantic Segmentation Network With Deep Seeded Region Growing, CVPR 2018 [web][code]

• Adversarial Complementary Learning for Weakly Supervised Object Localization, CVPR 2018

• Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation, CVPR 2018

• Multi-Evidence Filtering and Fusion for Multi-Label Classification, Object Detection and Semantic Segmentation Based on Weakly Supervised Learning, CVPR 2018

• Weakly Supervised Semantic Segmentation using Web-Crawled Videos, CVPR 2017(Spotlight) [web]

• Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach, CVPR 2017

• WILDCAT: Weakly Supervised Learning of Deep ConvNets for Image Classification, Pointwise Localization and Segmentation, CVPR 2017 [web][code]

• Learning random-walk label propagation for weakly-supervised semantic segmentation, CVPR 2017(Oral)

• Combining Bottom-Up, Top-Down, and Smoothness Cues for Weakly Supervised Image Segmentation, CVPR 2017

• Weakly Supervised Semantic Segmentation Using Superpixel Pooling Network, AAAI 2017

• Learning from Weak and Noisy Labels for Semantic Segmentation, PAMI 2017

• Learning to Segment Human by Watching YouTube, PAMI 2017

• Seed, Expand, Constrain: Three Principles for Weakly-Supervised Image Segmentation, ECCV 2016 [code]

• Backtracking ScSPM Image Classifier for Weakly Supervised Top-down Saliency, CVPR 2016, TIP 2018 Version

• Constrained Convolutional Neural Networks for Weakly Supervised Segmentation, ICCV 2015 [code]

• From Image-level to Pixel-level Labeling with Convolutional Networks, CVPR 2015

Resource

• Yunchao Wei talk in Chinese about WSL with image label

Arxiv paper

• Learning to Exploit the Prior Network Knowledge for Weakly-Supervised Semantic Segmentation, Arxiv1804

• Weakly Supervised Object Discovery by Generative Adversarial & Ranking Networks, Arxiv 1711

3.1 Deep activation

Propagate method	Papers
Global Max Pooling(GMP)	Is object localization for free? - Weakly-supervised learning with convolutional neural networks,CVPR 2015
Global Average Pooling(GAP)	Learning Deep Features for Discriminative Localization CVPR 2016
Log-sum-exponential Pooling(LSE)	ProNet: Learning to Propose Object-specific Boxes for Cascaded Neural Networks,CVPR 2016
Global Weighted Rank Pooling(GWRP)	SEC ECCV 2016
Global rank Max-Min Pooling(GRP)	WILDCAT, CVPR 2017

3.2 Weakly supervised Detection / Localization(TODO)

• Weakly Supervised Region Proposal Network and Object Detection, ECCV 2018

• TS2C: Tight Box Mining with Surrounding Segmentation Context for Weakly Supervised Object Detection, ECCV 2018

• Zigzag Learning for Weakly Supervised Object Detection, CVPR 2018

• W2F: A Weakly-Supervised to Fully-Supervised Framework for Object Detection, CVPR 2018

• Generative Adversarial Learning Towards Fast Weakly Supervised Detection, CVPR 2018

• Min-Entropy Latent Model for Weakly Supervised Object Detection, CVPR 2018 , PAMI19, [code]

• Weakly Supervised Cascaded Convolutional Networks, CVPR 2017

• Multiple Instance Detection Network with Online Instance Classifier Refinement, CVPR 2017 [code]

4.Other supervision

Points

• Deep Extreme Cut: From Extreme Points to Object Segmentation, CVPR 2018 [web][code]
• What's the Point: Semantic Segmentation with Point Supervision, ECCV 2016 [web][code]

Scribbles

• Normalized Cut Loss for Weakly-supervised CNN Segmentation, CVPR 2018
• ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation, CVPR 2016
• Learning to segment under various forms of weak supervision, CVPR 2015

5.Close Related or unpublished work

• PCL: Proposal Cluster Learning for Weakly Supervised Object Detection, Arxiv1807 [code]

• WebSeg: Learning Semantic Segmentation from Web Searches, Arxiv1803

• On Regularized Losses for Weakly-supervised CNN Segmentation, Arxiv1803

• Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignment, CVPR 2018

• Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptation, CVPR 2018

• Weakly Supervised Salient Object Detection Using Image Labels, AAAI 2018

• Weakly Supervised Object Localization on grocery shelves using simple FCN and Synthetic Dataset, Arxiv 1803

• Learning Semantic Segmentation with Diverse Supervision, WACV 2018

If some related works are missed, please kindly notice me by dxzhang@zju.edu.cn